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Abstract 

Involution words are variations of reduced words for involutions in Coxeter groups, first stud¬ 
ied under the name of “admissible sequences” by Richardson and Springer. They are maximal 
chains in Richardson and Springer’s weak order on involutions. In this paper, we initiate the 
study of involution words as an object of independent interest. To investigate their enumerative 
properties, we define involution analogues of several objects associated to permutations, includ¬ 
ing Rothe diagrams, the essential set, Schubert polynomials, and Stanley symmetric functions. 
These objects have geometric interpretations for certain intervals in the weak order on involu¬ 
tions, which we refer to as the geometric cases. In these cases, our definition for “involution 
Schubert polynomials” can be viewed as a Billey-Jockusch-Stanley type formula for cohomology 
class representatives of O n - and Sp 2n -orbit closures in the flag variety, defined inductively in 
recent work of Wyser and Yong. As a special case of a more general theorem, we show that the 
involution Stanley symmetric function for the longest element is a product of staircase-shaped 
Schur functions in both geometric cases. We prove, as an application, that the number of in¬ 
volution words for the longest element in the symmetric group is the dimension of a certain 
irreducible representation of a Weyl group of type B. 

This paper is the first in a series of three papers on involution words. It focuses primarily 
on algebraic tools related to involution words. In the second paper, we study the combinatorics 
of objects we call “atoms,” which are central to the study of involution words. The third paper 
focuses on the bijective combinatorics of involution words, including a new insertion algorithm. 
All three papers can be read independently. 
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1 Introduction 

Let (IP, S) be a Coxeter system and define X = X(W) = {x £ W : x = x -1 } as the set of involutions 
in IP. A reduced word for an element re € IP is a sequence (si, S2, ■ ■ ■ , Sk) with Sj G S of shortest 
possible length such that w = s\S 2 - ■ ■ Sk- An involution word for an element z € X is a sequence 
(si, S 2 , • • •, Sk) with Si E S of shortest possible length such that 

2 = (' ' ' ((1 X Si) X S 2 ) X • • • ) * Sk (1.1) 

where for g € VP and s £ S we let g x s be either gs (if s and g commute) or sgs (if sg ^ gs ). Note 
when g E X that g x s is also an involution. Less obviously, every z £ X has at least one involution 
word; for example, the empty sequence 0 is the unique involution word of 1 £ X. We write 7 Z(w) 
for the set of reduced words of re € IP and 7 Z(z) for the set of involution words of z £ X. More 
generally, given any pair of involutions y,z € X, we define 7 Z(y, z) as the set of sequences in S 
which, when appended to involution words for y, produce involution words for 2. The set 7 Z(y, z) 
may be empty, and we refer to its elements as involution words from y to z. 

Involution words display many properties analogous to those of ordinary reduced words, which 
accounts for our terminology. In particular, reduced words correspond to maximal chains in IP 
under the right weak order, while involution words correspond to maximal chains in X under the 
involution weak order defined by Richardson and Springer in (471 §3.17]. For initial intervals (that is, 
intervals starting at y = 1), involution words are the same as what Hultman calls “5-expressions” in 
[271 ,28] and are the right-handed versions of “admissible sequences” in [47,08] and “I*-expressions” 
in [431 [44 ]. For permutations, the involution weak order can be identified with the weak order on 
the set of 77-orbit closures in certain spherical varieties, and involution words are studied in this 
form by Can, Joyce, and Wyser in ESI E!- Specifically, the orbits induced by the actions of the 
orthogonal and symplectic groups on the flag variety have weak orders whose chains correspond to 
involution words in the intervals starting at 1 and v n = [2,1,4, 3,... , 2 n, 2n—1] € S 2 n , respectively. 
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These geometric cases are of particular interest, and lead us to define, alongside 77(y), the set 

7^fpf(~) = f T^(v n , z) for 2 € X(5 2n )- (1.2) 

Abusing notation, elements of this set will be called fixed-point-free involution words, since v n is 
the unique minimal involution with no fixed points in S^n- The set 77 fpf(z) will be non-empty if 
and only if z € I(S 2 n ) has no fixed points, in which case every involution in the interval between 
v n and z in weak order will also be fixed-point-free. 

Before describing our work on involution words, we provide a brief overview of the geometry 
underlying the geometric cases. Let B be the Borel subgroup of lower triangular matrices in GL n (C) 
and denote by Fl(ra) = R\GL n (C) the flag variety. The right orbits of the opposite Borel subgroup 
B + in Fl(n) decompose into Schubert cells, whose Zariski closures are the Schubert varieties X w , 
which are indexed by permutations w £ S n , and which can also be defined explicitly using a fixed 
reference flag and rank conditions determined by w. By instead considering the right actions on 
Fl(n) of another group K, such as the orthogonal group O n (C) or (when n is even) the symplectic 
group Sp n (C), one obtains different orbit decompositions. These A"-act ions decompose Fl(n) into 
A'-orbits Yy indexed by arbitrary involutions y £ T(S n ) when K = O n (C) and by fixed-point-free 
involutions in S n when K = Sp n (C). Again, Yy can be defined using a fixed reference flag and 
explicit rank conditions determined by y. 

Each Schubert variety determines a class [X w ] in the cohomology ring H*(F\(n), Z), which can 
be identified with the quotient of Z[aq,..., x n ] by the ideal generated by the symmetric polynomials 
of positive degree via the Borel isomorphism (see (12.91) 1. The Schubert polynomial 6 W , defined by 
Lascoux and Schiitzenberger [32], is a particularly nice choice of representative under this map for 
the cohomology classes of the Schubert variety X w . For A'-actions where K = O n (C) or Sp n (C), 
the cohomology classes of Yy in H*(F\(n), Z) map to the same quotient of Zfaq,..., x n ]. For 
these classes [Y y A ], Wyser and Yong have defined similarly nice polynomial representatives T y, 
which they call T-polynomials [[55]. The construction of Y, A in [55 j relies on first choosing a 
representative for the class of a point, and then showing compatibility with certain compositions 
of divided difference operators. 

In this paper, we initiate the study of involution words from an enumerative perspective. To do 
so, we develop many analogues of combinatorial objects used in the study of reduced words. Most 
notably, we introduce analogues of Rothe diagrams and Fulton’s essential set for the geometric 
cases and analogues of Schubert polynomials and Stanley symmetric functions for all involution 
words. Many of these definitions are simple extensions of the ordinary versions, especially in light 
of the following result, which is a consequence of [271 Lemma 3.16]. 

Theorem-Definition 1.1. For each y,z £ T{W), there exists a finite subset A(y,z) C W such 
that lZ(y,z) = U weA(y z)'^( w ) ■ Equivalently, every involution word from y to z is a reduced word 
for some element of W and the set 7 Z(y,z) is closed under the braid relations for (IF, S). For 
y,z £ X and w £ A(y, z), we say w is a relative atom from y to z. 

In the geometric cases, we define A(y) = .4(1, y) and Mfpf(^) = A(v n ,z). 

Remark. The theorem-definition follows from results of Richardson and Springer in [47, §3]. A 
more direct, general proof using our present notation appears in our complementary work [23]. 
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In both geometric cases, we introduce the involution Rothe diagrams D(y) and Z) F pf (y) of y E X 
as certain restrictions of the usual Rothe diagram D(y). We then define the essential sets Ess(-D(y)) 
and Ess(-Dfpf(?/)) as sets of southeast corners in the corresponding involution diagram. This closely 
mirrors the definition of Fulton’s essential set Ess (D(w)) for w E S n . In Proposition 13.161 we show 
that the involution essential sets determine a subset of the rank conditions sufficient to define 
when K = O n (C) or K = Sp n (C), respectively. The proof is largely a consequence of the analogous 
result for the Reaction, with some subtleties in the fixed-point-free case. These objects prove to 
be a key tool in our study of involution Schubert polynomials and involution Stanley symmetric 
functions. 

As discussed previously, Schubert polynomials were originally defined using divided difference 
operators. However, they can also be viewed as a sort of generating function over reduced words. 
More specifically, Billey, Jockusch and Stanley [2] and Fomin and Stanley m showed the following 
explicit combinatorial formula. Let s, denote the simple transposition (i,i + 1), so that S n is a 
Coxeter group relative to the generating set {si, S 2 , • • •, s n _i}. Fix w € S n . and for each a = 
(s ai ,s a2 ,..., s ak ) E TZ(w), let C(a) be the set of sequences of positive integers I = (i\,i 2 ... ,ik) 
satisfying 

ii Y i 2 < ■ ■ ■ < ik and ij < ij + i whenever aj < Oj+i. 

We write I < a to indicate that ij < aj for all j and define xj = x'* 2 • • • Xi k . The Schubert 

polynomial corresponding to w E S n is then given by 

&w = ^2 ^2 XI E Z[xi,... ,x n ]. (1.3) 

a£R,(w) /GC(a) 

I<a 

This formula makes clear that 6 W is homogeneous with degree equal to the length of w. Similarly, 
the Stanley symmetric function of w is 

*■-- E E xi E Z[[xi,x 2 , • • • ]] (1.4) 

ag 7Z(w) l€C(a) 

(this definition is actually F w -i in [5DJ)- These were introduced by Stanley to help enumerate 
reduced words, since the coefficient of x\x 2 ■ ■ ■ xu w ) in F w is |72.(io)|. Note F w = lirrgv^oo &i N xw 
where In x w denotes the image of w under the natural embedding S n ^ Sn x S n C Sn+u and 
the limit is taken in the sense of formal power series. This limit is called stabilization, and Stanley 
symmetric functions are sometimes referred to as stable Schubert polynomials. 

For y,z E Z(S n ), we define involution Schubert polynomials and involution Stanley symmetric 
functions analogously as 

& y ,z = ^2 ~52 Xl and F y , z = Y\m°& lN x y ,i N xz= ^2 ^2 X Y 

a£Ti(y,z) l£C(a) aGTZiy^) l£C(a) 

I<a 


respectively. Note by Theorem-Definition O that & y>z = J2weA(y,z) and F y,z = A{y,z) 
For the geometric cases, we define 


G y d = 6 


1 , 2 / > 


F — f F 
r y ~ ^1,2/! 


(~FPF def A, 

'Oy — ^v n ,y: 


and 


g,FPF def jj, 

X y ~ ^v n ,y 


As one would hope, these involution Schubert polynomials are the same (up to scaling factor) as 
Wyser and Yong’s representatives for [Y y A ]. Let n{y) be the number of two-cycles in y E Z{S n ). 
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Theorem 1.2. For each y £ F(S n ) and each fixed-point-free 2 £ X(S , 2 n ), it holds that 

2 K ^& y = and <5 FPF = T® Pz ". 

In either geometric case, the longest permutation w n = [n, n—1,..., 1] £ S n indexes the orbit 
of the fixed reference flag, i.e., the class of a point. For this class, Wyser and Yong’s polynomial 
representatives are 

= II ( X i+ X 3 ) aIld T ^2n n = 6^ P 2 F „ = II ( Xi+Xj ). (1.5) 

l<i<j<2n—i 

A permutation is dominant if it is 132-avoiding. We extend these product formulas to dominant 
involutions as follows. 

Theorem 1.3. Let y £ I(S n ) and let z £ I(S- 2 n ) be fixed-point-free. If y and z are dominant, then 

& y = JJ (xj + Xj) and 6 FPF = (Xi + Xj ) 

(i,j)eD(y) (iJle^FPFfz) 

where D(y ) and D FFF (z) are defined as in Section Edl 

Theorem [T3] is restated as Theorem 13.261 and is a special case of Theorem l3.27l which describes a 
simple formula for the involution Schubert polynomials of a more general class of permutations that 
we call weakly dominant. This formula is the product of & Wk and a specialization of the double 
Schubert polynomial of some v £ S n , where k and v are determined by the weakly dominant 
involution. 

In [50], Stanley showed that Stanley symmetric functions F w are symmetric, and computed 
several functions explicitly, most notably showing that F Wn is the Schur function ss n indexed by 
the staircase shape partition S n = (n — 1, n — 2,..., 1). This implies that \TZ(w n )\ is equal to 
f Sn , the number of standard Young tableaux of shape 5 n . As a consequence of work by Lascoux 
and Schiitzenberger [S3], and as proven bijectively by Edelman and Greene ED, Stanley symmetric 
functions are Schur positive, i.e., can be expressed as positive integer sums of Schur functions. Since 
involution Stanley symmetric functions are sums of Stanley symmetric functions, they inherit this 
property. In the geometric cases, we characterize the involutions whose involution Stanley symmet¬ 
ric function is a single Schur function. Moreover, by carefully studying the stabilization of certain 
weakly dominant involution Schubert polynomials, we obtain expressions for the corresponding in¬ 
volution Stanley symmetric functions. Most notably we derive the following, which was conjectured 
in 2006 in unpublished work of Cooley and Williams ED]. 

Theorem 1.4. Let p = and q = and set P = ( 2 ), Q = ( 9 ), and N = (”). Then 

Fw n = s Sp s Sq and F fpf = (s 5 J 2 . 

Consequently, \ti(w n )\ = ( P p Q )f Sp f 5q and \'R. FFF (w 2n )\ = ( 2 ^) (f Sn ) 2 - 

Theorem [L4l is a special case of Theorem 13.431 which provides product formulae for a certain 
family of weakly dominant involutions. Moreover, every involution Stanley symmetric function 
computed in Theorem 13.431 is Schur-P positive. In forthcoming work [24] . we present a bijective 
proof that F y = F\ y is Schur-P positive for all y £ X{S n ). We do not yet have a good understanding 
of when the symmetric function F y z is Schur-P positive for arbitrary involutions y,z £ I(S n ). It can 
happen that an involution Stanley symmetric function is not expressable using Schur-P functions. 
For example P[ 2 , 1 , 3,41 ,[ 3 , 4 , 1,21 = s (i,i)> which is not in the ring generated by Schur-P functions. 
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Question 1.5. For which y,z € T(S n ) is F yz Schur-P positive? 

Conjecture 1.6. For y G T(S n ) fixed-point-free, F y PF is Schur-P positive. 

Although our enumerative results are restricted to the geometric cases for the symmetric group, 
the objects we study have natural analogues in other Coxeter groups. Many tantalizing questions 
remain in this direction. For example, Haiman showed in m that \TZ(w^)\ = where w% 

is the longest element in the Weyl group B n and n n = (n, n,...,n). Computations suggest the 
following version of this theorem for involution words. 

Conjecture 1.7. The set TZ(w^) has size f Sn+1 . 

Additionally, there is a notion of twisted involution words for which Schubert polynomial and 
Stanley symmetric function analogues are readily defined. We do not explore these objects in 
the present paper, but it remains a question of great interest to find geometric interpretations for 
twisted involtution Schubert polynomials. 

Before outlining the structure of the paper, we provide a brief discussion of the relationship 
between our work and that of Wyser and Yong [55]. Our proof of Theorem 11.21 proceeds by 
generalizing a characterization of Schubert polynomials to the involution setting. This allows us to 
show our formula coincides with Wyser and Yong’s formula for the class of a point and behaves in 
the same way with respect to divided difference operators. However, a general formula of Brion [5j 
Theorem 1.5] shows that (up to a power of 2) [Y y K ] is a multiplicity-free sum of Schubert classes 
Ei veA K (y)[Xw]- Wyser and Yong [55, Section 5] note that their representative for [Y y x ] is a linear 
combination of Schubert polynomials, and therefore is equal to Yh w &A K (y) ®w (again up to a power 
of 2). One then immediately gets an analogue of (11.311 for Wyser and Yong’s representatives by 
replacing 7 Z(w) with U™e.4(y) w ) ( in the K = ° n (C) case) or LLsApfO) ( in the K = S Pn( c ) 

case). From this point of view, the main contribution of Theorem [L2] is combinatorial: we identify 
Ui ueA K {y) T^( w ) as ( or 7 ^-fpf {y)) and A K (y) as the set A(y) (or Mfpf(j/)) defined in Theorem- 

Definition P While the latter is also done in [7], the connection is mostly left implicit. 

As mentioned earlier, Theorem ra is a special case of Theorem 13.271 which describes certain 
involution Stanley symmetric functions as the product of a Schubert polynomial and the specializa¬ 
tion of a double Schubert polynomial. In Section (3[ we present a proof relying on carefully studying 
the image of & Wn and under certain divided difference operators. However, by considering 

the intersection of appropriate varieties corresponding to these polynomials, one obtains a different 
characterization of the In Appendix [A] we present an alternate proof of Theorem 13.271 

based on these geometric considerations. This approach is quite different from that of Wyser and 
Yong. Indeed, it provides a new proof of their product formulae (11.51) for & Wn and <3^ and that 
the involution Schubert polynomials are cohomology representatives, which is independent both of 
their work and Brion’s. However, this approach does not recover the fact, proved by Wyser and 
Yong, that these polynomials are equivariant cohomology representatives. 

The remainder of the paper is structured as follows. Section [2] describes the previous under¬ 
standing of involution words necessary for this paper. It then recalls definitions and known results 
for Rothe diagrams, Schubert polynomials, and Stanley symmetric function required to prove our 
results, as well as a more detailed discussion of the geometric connections. In Section [3J we prove 
the main results of this paper. This requires defining many objects, including involution diagrams, 
and developing significant background on techniques such as stabilization. Finally, Appendix 12.11 
presents a geometric proof of Theorem 13.271 
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2 Preliminaries 

Write P = {1, 2, 3,... } for the positive integers and define N = {0} U P and [n] = {i € P : i < n}. 
If (W, S ) is a Coxeter system, then we write £ : W —> N for its length function, and denote by 

Des l(w) = {s £ S : £(sw) < £(w)} and Des r(w) = {s £ S : £(ws ) < £(w)} (2-1) 

the left and right descent sets of an element w £ W. 


2.1 General properties of involution words 

Here we review the basic properties of involution words for an arbitrary Coxeter group. Most of 
this material appears in some form in [171 .45, Ufl or the more recent papers [26} [27 , 128] . 

Remark. Our definition of involution words has a straightforward generalization to twisted invo¬ 
lutions in Coxeter groups, by which we mean elements w £ W satisfying w^ 1 = w* for some fixed 
5-preserving automorphism * of W of order two. This more flexible setup is the point of view of 
our references, but our present applications will not require this generality. 

Let (IT, 5) be any Coxeter system and write Z = Z(W) = {w £W : w ” 1 = w}. 

Remark. Recall that for y £ Z and s £ S we define 


y xi s = 



if ys / sy 
otherwise. 


Although (y xi s) x s = y for s £ S, the operation xi : Z x 5 —> Z usually does not extend to a 
right W -action. For example, if s,t £ S are such that sts = tst then ((1 X s) x t) xi s = t but 
((1 xi t) xi s) x t = s. Nevertheless, we usually omit all parentheses in expressions like (11.11) . 

Define 7 Z(w) for w £ W and 7Z(z) for z £ Z as in the introduction. Recall that for y,z £ I, the 
set lZ(y,z) consists of all words (si,...,Sfc) with Si £ S such that for some (equivalently, every) 
word (?’i,... ,rj) £ 7 Z(y) it holds that 

(n, .. .,rj,s i,... ,s k ) £ 7 Z(z). 

Call 7 Z(y,z) the set of (relative) involution words from y to z. Note that lZ(y,y) = {0} where 0 
denotes the empty sequence, and that 7 Z(y, z) may be empty, for example if y exceeds 2 in length. 

Fix y £ Z and s £ S. It is a consequence of the exchange principle that £(sys) = £(y) if and 
only if sys = y m Lemma 3.4], and so if s £ Desjt(y) then 


i(y xi s) 


£(y) — 2 if y x s = sys 
£(y) — 1 if y x s = ys. 


( 2 . 2 ) 
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From this property, it follows by induction on length that 7 Z(y) ^ 0 for all y £ X, so we may set 

£(y) the common length of all involution words for y £ X. (2-3) 

We abbreviate by writing £(y, z ) to denote the difference £(z) — £{y). 

Remark. The map £ : IF —> N is denoted L in [37J §3] and p in 2T>, EZl ESj- Incitti [2TJI 3DJ has 

derived useful combinatorial formulas for £ when W is a classical Weyl group. In the case when 
W = S n is a symmetric group, one has 

Kv) = \ (t(y) + «(y)) for y e i{S n ) 

where £(y) is the usual length and k( y) is the number of 2 -cycles of an involution y. 

Write < for the (strong) Bruhat order on IF. Recall that this is the partial order in which u < v 
if and only if in each reduced expression for v one can omit a certain number of factors to obtain 
a reduced expression for u. Thus u < v implies £(u) < £(v), and it follows from (12.21) that if y £ X 
and s £ Des n(y) then y x s < ys < y. There is a close relationship between the the Bruhat order 
on X and involution words, which was one motivation for their study originally in [4TI 35] , For 
example, (X, <) is a graded poset with rank function £ : X —> N [26, Theorem 4.8], and this poset 
inherits the subword characterization of (IF, <) given above, but with the role of reduced words 
replaced by involution words }28l Theorem 2.8]. From these results, it is clear that if y £ X and 
s € S then the following are equivalent: 

y X S < y £(ys) = £{y) - 1 l(y yi s) < £{y) £(y x s) = £{y) - 1. 

These properties imply the following useful alternative definition of the set lZ(y,z): 

Lemma 2.1. If y, z € X, then a word (si, S 2 ,..., Sk) with s, € S belongs to 7 Z(y, z) if and only if 

y < w\ < W 2 < ■ ■ ■ < Wk = z where Wi = y x si x S 2 x ■ ■ ■ x s*. 

Recall that 7^ F PF(y) = 7^.(u n , y) where v n = s\s^ - ■ ■ S 2 n+i- 

Corollary 2.2. If y £ X(S , 2 n ) then lZ FPF (y) is non-empty if and only if y is fixed point free. 

Proof. Note if y € S^n is a fixed-point-free involution and s is a simple transposition that sys is 
also fixed-point-free while y x s = ys only if s £ Des r{w)\ then invoke the preceding lemma. □ 

Recall the definition of A(y, z) for y,z € X(W) from Theorem-Definition 11.11 The properties of 
A(y, z) are one of the main subjects of our work [23]. Call A(y, z) the set of (relative) atoms from 
y to z, and recall A(y) = A(l, y). While A(y, z) may be empty, for example when £(z) < £(y), the 
set A(y) is always nonempty, and we always have .4(1) = A(y,y) = {1}. It is clear that 

A(y,z) = {u: € IF : £{w) = £(y,z) and vw £ A{z) for some v £ A(y)}, 

so A(y, z) can be computed from A(y) and A(z). 

Example 2.3. For the involutions V 2 = [2,1,4,3] and uq = [4,3,2,1] in S 4 we have 

A(w a ) = {[ 2 ,4,3, 1 ], [3,4, 1 ,2], [4, 2 ,1,3]} and A(v 2 , w A ) = {[1,3,4, 2], [3, 1 , 2 , 4]}. 

In general, the sets A(w n ) and A(v n ,W 2 n ) have cardinality (n — 1)!! and n! and are given by a 
simple recursive construction due to Can, Joyce, and Wyser mm- 


The essential properties of atoms are summarized by the following statement, which is proved 
as [23j Proposition 2.8]. 

Proposition 2.4 (See [23] 1. Let y,z £l and s G S. 

(a) If s £ Des#(z) then A(y, z) = {res : w G A(y, z x s) and s G Des r(w)}. 

(b) If s G Des n(y) then A(y,z) = {sre : w G A(y xi s,z) and s G Desire)}. 

Consequently, if u G A(y, z) then Des#(u) C Desij(^) and Des l(u) C S\ Des n(y). 

We mention another order on I which will be of relevance. Recall that the left and right weak 
orders <l and <r on W are the transitive closures of the relations w <l sw and w <r wt for 
w G W and s,t G S such that £(sw) > £(w) and £{wt) > £{w). Following [23 Section 5], we dehne 
the (two-sided) weak order <t on X as the transitive closure of the relations 

w <t rr xi s for w G X and s G S such that £{w) < £(w xi s). 

Evidently A(y, z) is nonempty if and only if y <t z , and each element of 1Z(y , z) corresponds to 

a maximal chain from y to z in the poset ( X , <t)- If y, z G X then y <t z implies y < z, but the 
reverse implication does not hold in general. 

2.2 Diagrams and codes for permutations 

We write Soo for the group of permutations w of P whose support supp(ie) = {* G P : w(i) ^ i } 
is finite, and identify S n for n G P as the subgroup of permutations w G 5oo with supp(w;) C [n]. 
Recall that the right descent set of w G S^, dehned as in (12.lj) with respect to the generating set 
of simple transpositions {sj = (i,i + 1) : i G P}, is given more explicitly by 

Desn(w) = {sj : i G P and w(i) > w(i + 1)}. (2.4) 

We say that i is a descent of w G if w(i) > w(i + 1), so that s* G Des^(u;). 

Recall (e.g., from [41j §2.1.1]) that the Rothe diagram of w G Soo is the set of positions 

D(w) = {(z, j) G P x P : j < w(i ) and i < u; _1 (j)} . 

Note that D(w) is obtained by applying the map (i. j) ( i,w(j )) to the inversion set of w. 

Consequently D(w^ 1 ) = D(w) T where T denotes the transpose map (i,j) (j, i), and if w G S n 

and has largest descent k, then D(w) C [k] x [n]. 

Example 2.5. We have D(y n ) = {(2 i — 1,2* — 1) : i G [n]} and D(w n ) = {(i,j) G P 2 : i + j < n}. 

The diagram of an integer partition A = (Ai > A 2 > ...) is the set {(i,j) G P x P : j < A.;}. We 
often identify partitions with their diagrams, and write (i,j) G A to indicate that (i,j) belongs to 
the diagram of A. If A and p, are partitions with p C A then the skew shape X/p is the complement 
of the diagram of p in the diagram of A. The shifted shape of a strict partition A (i.e., a partition 
with distinct parts) is the set {('i, j + i — 1) : (i, j) G A}. Two finite subsets of P x P (in particular, 
Rothe diagrams or diagrams of partitions or skew shapes or shifted shapes) are equivalent if one 
can be transformed to the other by permuting its rows and then its columns. 
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Example 2.6. If A = (2,2,1) and p = (1) then {(1,1), (1, 3), (2,3), (3,1)} is equivalent to X/p. 

The code of w £ S n is the sequence c(w) = (ci(w), c 2 (w),... ,c n (w )) £ N n where 

Ci(w) = \{j £ [n] : i < j and w(i ) > u>(j)}\. (2.5) 

Observe that Ci(w ) is the number of cells in the z th row of D(w). The shape A (w) of w £ S n is the 
partition of i{w) whose parts are the nonzero entries of c(w). 

Example 2.7. If w = [3, 7, 4,1, 6, 5, 2] then c(w) = (2,5, 2, 0, 2, 1, 0) and A (w) = (5, 2,2, 2, 1), 


2.3 Schubert polynomials 

We sketch here the fundamental properties of the Schubert polynomials 6 W as defined in the intro¬ 
duction; our main references are the texts mum and papers We write 

V n = Z[xi,X 2 ,...,x n ] and Voo = Z[aq, x 2 ,... ] 

for the rings of polynomials in finite and countable sets of commuting variables {xi,x 2 , ■ ■ ■ }. The 
group S n (respectively, S 0G ) acts on V n (respectively Voo) by permuting variables. With respect to 
this action, the divided difference operator di for i £ P is defined by 


dif = (/ - Sif)/(xi - Xi+i) for / 6 Voo- 

For example, di(xf) = xf + XiXi+i + xj +1 . It is a standard exercise to check that this formula in 
fact gives a linear map <9* : Voo ~> Voo, and that di(fg) = f ■ dig whenever Sif = /. 

We first observe, following the notes of Knutson n how one may characterize the Schubert 
polynomials without explicitly constructing them using the divided difference operators. 

Theorem 2.8 (See [32]). The Schubert polynomials {© U i}ioe5oo are the unique family of homoge¬ 
neous polynomials indexed by the elements of Soo such that 


©i = l and di& w 


&w Si if Si € Des R (w) 
0 otherwise 


for all i 6 P. 


The divided difference operators satisfy df = 0 as well as the Coxeter relations for Soo given by 

didi+idi = di + \didi + \ and didj = djdi for i,j 6 P with \i — j\ > 1. (2.6) 

For w £ Soo, we m ay thus define d w = d^di 2 ■ ■ ■ di k for any reduced word (sj 15 Sj 2 ,..., s %k ) € 7 Z(w). 
Recall that w n denotes the longest element of S n . 

Theorem 2.9 (See [H]). If n £ P and v £ S n then 

& w n = x r f~ l x^ 2 x1~‘ A ■ ■ ■ x n _i and 6 V = d v -i Wn <5 Wn . 

Let y = {yi,y 2 , ... } be another countable set of commuting variables, which commute also with 
x = {xi,x 2 , ... }. If / £ Voo then we write f(y) to denote the polynomial given by evaluating / at 
Xi = yi, and for emphasis we sometimes write / = f(x). We let Voo(x;y) = Z[xi, y\, x 2l y 2 , ... ] be 
the polynomial ring in x and y together, and take m Proposition 2 . 4 . 7 ] for the following definition: 
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Definition 2.10. The double Schubert polynomial corresponding to w £ Soo is 

&w{x-y)= ^ &u(x)G v {-y) € Voofay). 

W=V~ 1 U 

£(w)=£(u)+£{y) 

Let S 0 o act on Voo (x',y) by permuting only the Xi variables, and extend the formula for di to 
an operator V oc (x', y) —> Voo{x',y) with respect to this action. The following then holds: 

Theorem 2.11 (See [41 jh If n € P and v £ S n then 

&w„(x;y) = JJ and & v (x; y) = d v -i Wn 6 Wn (x; y) 

where the product on the left is over i. j £ P. In particular, & v = & v (x;0). 

If w £ S n then 6 W is a polynomial in at most n — 1 variables, though often fewer. Explicitly, 
we note the following useful fact, which appears as m Proposition 2.5.4]. 

Proposition 2.12 (See |41]L The set of Schubert polynomials & w with w £ Soo ranging over all 
permutations with largest descent at most n forms a basis for V n over Z. 

A permutation w £ S n is dominant if it is 132-avoiding, or equivalently if its Rothe diagram is 
the diagram of a partition (see [34l Exercise 2.2.2]). A permutation w £ S n is Grassmannian if it 
has at most one right descent, or equivalently if for some r > 0 it holds that c\{w) < • • • < c r (w) and 
Ci[w) = 0 for all i > r. For permutations of these types, the corresponding Schubert polynomials 
have the following formulas, which appear m Propositions 2.6.7 and 2.6.8]. Here, recall that s\ 
denotes the Schur function indexed by a partition A. 

Proposition 2.13 (See }41jl. Let w € Soo- 

(a) If w is dominant then 6 w (x-y) = Y\(ij)eD(w)( x i ~ %')■ 

(b) If w ^ 1 is Grassmannian with unique descent r, then & w = sx( w )( x i, ■ ■ ■ ,x r )- 

2.4 Cohomology of flag varieties 

We review the geometric context that lead to the consideration of Schubert polynomials. Let Fl(n) 
denote the set of complete flags F, = (0 = Fq C F\ C ■ ■ ■ C F n = C n ), where each E) is a subspace 
of dimension i, given the structure of a projective algebraic variety via the Pliicker embedding as 
in [HI §3.6.1], We identify Fl(n) with the right coset space B \ GL n (C), where B is the Borel 
subgroup of lower triangular matrices in GL n (C). 

The general linear group GL n (C) acts on the right on Fl(n) by multiplication. Let B + = 
w n ■ B ■ w n denote the Borel subgroup opposite to B , consisting of the upper triangular matrices 
in GL n (C). It follows from the Bruhat decomposition of GL n (C) that the distinct orbits of B + on 
Fl(n) identified with B \ GL n (C) are given by B\BwB + for w £ S n , where S n is embedded as the 
subgroup of permutation matrices in GL n (C). Define 

X v , = B\BwB + and X w = B\BwB + for w £ S n 

where on the right the bar denotes the Zariski closure. We call X w the Schubert cell attached to 
w £ S n and X w the corresponding Schubert variety. 
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Remark. Because we identify Fl(n) with B\GL n (C) rather than GL n (C)/R and define Schubert 
cells as right B + -orbits rather than left B-orbits, our definitions differ from those in [ fll §3.6] by a 
transformation of indices. Explicitly, the sets £l w for w £ S n which Manivel refers to as Schubert 
cells are given in our notation by = X WnW ■ w n whence X w = w n ■ F, WriW . What we call X w is 
related to Manivel’s dehnition of the Schubert variety of w £ S n by the same transformations. It 
thus follows from [41, §3.6.2] that X w is an irreducible variety of codimension l(w) in Fl(n). 

It will be useful to recall the following concrete description of Schubert cells and varieties. 
Choose a basis ei,e 2 ,...,e n of C n for each j £ [n] define Vj = C-span{ei,e 2 Given a 
vector space U C C n , we write proj : U Vj for the restriction to U of the usual linear projection 
C" —> Vj mapping e* 0 for i > j. Also define 

rk w (i,j) = \{t £[i\: w(t ) G [j]}[ for w £ S n and i,j £ [n]. (2.7) 

By m Proposition 3.6.4] (noting the remark above), we then have 

X w = {F. £ Fl(n) : dim (proj : F t -£ Vj) = rk w (i,j) for each i,j £ [n]} , ^ 

X w = {F, £ Fl(n) : dim (proj : F. L —y Vj) < rk w (i,j) for each i, j £ [n]} . 

These conditions say that F, belongs to X w (respectively, X w ) if and only if for each i. j £ [n], the 

upper left i X j submatrix of a matrix representing F, has rank equal to (respectively, at most) the 

number of l's in the upper left i x j submatrix of the permutation matrix of w. 

If A is a smooth complex algebraic variety and V is a closed subvariety, then there is a corre¬ 
sponding cohomology class [V] £ ff*(X,Z), with the important property that [V FI W ] = [F] [IF] 
when V and IF intersect transversely on an open subset of V fl IF. When X is compact one defines 
[F] by first triangulating F to obtain a homology class and then taking its Poincare dual. In gen¬ 
eral, one can view [F] as the image of the class of F in the Chow ring of X under an appropriate 
map to H*{X)\ see [l6| or [HI Appendix A] or [HI Chapter 19]. 

For each Schubert variety X w C Fl(n) one obtains in this way a corresponding Schubert class 
[AH G H*(Fl(n),Z) which is denoted a w in [41, §3.6.3]. As in the introduction, we identify the 
Schubert classes with elements of the coinvariant algebra of the symmetric group via the Borel 
isomorphism (see m §3-6.4]) 

ff*(Fl(n),Z)4P n /(A+), (2.9) 

with (A+) denoting the ideal in V n generated by the symmetric polynomials of positive degree. 
Via these identifications, the divided differences d w for w £ S n make sense as an operators on 
i7*(Fl(n),Z), since di maps (A+) into itself. Bernstein, Gelfand, and Gelfand [3] show that 

o rv i f[X ws ] if s £ V>es R {w) 

o s [X w \ = < _ for w £ S n and s £ {s* : i £ P}. (2.10) 

I 0 otherwise 

Consequently, once one fixes a polynomial representing [X Wti ] (the class of a point), representatives 
for all [Am] are determined by induction. Lascoux and Schiitzenberger |35] have shown that the 
Schubert polynomials are representatives of the Schubert classes formed in precisely this way: 

Theorem 2.14 (Lascoux and Schiitzenberger [35]4. For all w £ S n it holds that 6 W = [X w \. 
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2.5 Stanley symmetric functions 

Let A = A(x) be the algebra of symmetric functions over Z in the variables x = {xi : i £ P}. 
We follow the standard conventions from m for referring to the various well-known bases of this 
algebra. 

Recall the definition of the Stanley symmetric function F w from Q. Stanley [50] was the first 
to consider this power series and prove that it belongs to A. In this section we review an alternate 
definition due to Edelman and Greene m which makes this fact more transparent and explains 
the connection between F w and the problem of counting reduced words. 

Remark. Following Lam |37l 138] . our conventions for F w differ from Stanley’s original definition by 
the transformation w £->■ re -1 ; m Corollary 2.2] is helpful for understanding these transformations. 

Let T be a (Young) tableau, i.e., an assignment of positive integers to the cells of the diagram of 
a partition (or, more generally, to the cells of some sequence of partitions or skew shapes or shifted 
shapes), called the shape of T. Say that T is strict if its entries are strictly increasing both from 
left to right in each row and from top to bottom in each column. A strict tableau is standard if its 
entries comprise the set [n] for some n £ N. The reverse reading word of T, denoted rrw(T), is the 
word obtained by reading the rows of T from right to left, starting with the top row. For example, 


1 

2 

3 

2 

OO 



has rrw(T) = (3,2,1,3,2). A tableau is reduced for w £ Sqq if it is strict and its reverse reading 
word is a reduced word for w, where we identify a sequence of positive integers (R, » 2 ,. • ■ ,ik) with 
the word (sq, Sj 2 ,..., Si k ). The tableau T above is reduced for w = S 3 S 2 S 1 S 3 S 2 = [4, 3,1, 2], Results 
of HD show that we may alternatively define the Stanley symmetric function F w as follows: 

Theorem 2.15 (Edelman and Greene [II]). If w £ S)*, then F w = Yl\ a w,\ s \ € A where the sum 
is over partitions A and a w> \ is the number of reduced tableaux for w of shape A. 

Recall that f x is the number of standard tableaux of shape A. 

Theorem 2.16 (Edelman and Greene [IT]). If w £ Soo then \lZ(w)\ = J2\ a w,\f X - 

Edelman and Greene OH provide bijective proofs of these identities using a variant of the RSK 
correspondence, now referred to as Edelman-Greene insertion. The latter map gives an algorithm 
for calculating F w for any w £ S Qc ; other, more efficient methods of computation are described in 
PH [221 Ha [39]. For our purposes, it will suffice to recall one exact formula [4j Proposition 2.4], 

Proposition 2.17 (Billey, Jockusch, Stanley HD- If the Rothe diagram of w £ S n is equivalent to 
a skew shape A//q then F w = s \is the corresponding skew Schur function. 

A permutation w £ S n is vexillary if it is 2143-avoiding or, equivalently, if its Rothe diagram is 
equivalent to the diagram of a partition m Proposition 2.2.7]. We also recall the following result, 
which derives from either [401 Eq. (7.24) (iii)] or [50, Theorem 4.1]. 

Theorem 2.18 (Macdonald [40j : Stanley [50]). The Stanley symmetric function F w is a Schur 
function if and only if w is vexillary, in which case F w = s\(w)- 

Example 2.19. The reverse permutation w n £ S n is vexillary with A (w n ) = S n , so F w = s$ n and 
via Theorem 12.161 we recover the result of Stanley [50] that \TZ(w n )\ = f Sn . 
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3 Involution words for symmetric groups 

As in Section I2TT1 we write Z(S OQ ) and Z(S n ) for the sets of involutions in and S n . In addition, 
we let Z FPF (S 2 n) be the set of fixed-point-free involutions in and set Z FPF (S oa ) = UneP-^ FPF (‘-’ 2n ) - 
Throughout this section, we write g n for the Grassmannian involution 

9n = (l,n)(2,n + 1 ) • • • (n, 2n) = [n + l,n + 2 ,... ,2 n, 1 , 2 ,... ,n] € X FP F (S 2n )- (3.1) 

Recall that n(y) is the number of 2-cycles in an involution y. We have 

Kv) = \ (t{v) + K (y)) and 4 pf (z) = \ (t(z) - k(z)) 

for y £ Z(Sn) and z £ Z PPP (S 2 n), where l(y) is as in (12.31) and £ FPF (z) = £(v n , z) = £(z) - n. 

3.1 Atoms for permutations 

Recall the sets A(y,z) from Theorem-Definition 11.11 As in the introduction, we write 

Apf(^) = A(v n , z) for 2 £ X fpf (5 2 „), 

where as usual v n = (1,2) (3,4) • • • (2n — 1, 2n), so that lZ PPP (z) = Uus^epfIz) u )• 

As noted earlier, Can, Joyce, and Wyser mm have recently studied the sets A(y) and A PPP (y) 
for y £ I(S' 00 ); they provide a useful set of conditions, involving only the one-line representations of 
permutations, which classify their elements. (Several left/right-handed conventions in [ 6 ) [7] are the 
mirror images of the ones we adopt here, so the elements in the sets described by the main results 
f7, Theorem 2.5 and Corollary 2.16] are actually the inverses of what we call atoms.) The sets A(y) 
and A FPF (y) may also be viewed as special cases of the sets W(Y) that Brion defines geometrically 
in m §i-i]. These sets of atoms have several special properties which do not generalize to other 
Coxeter groups, which we discuss in the complementary paper |23| . We require some results from 
that work, which we quote as follows. The following combines |231 Corollaries 6.11 and 6.23]. 

Theorem 3.1 (See [23]). Let y £ I(S' 00 ). Then |A(y)| = 1 if and only if y is 321-avoiding. Likewise, 
if y is fixed-point-free, then |A FPF (y)| = 1 if and only if y is 321-avoiding. 

We can identify one atom of any involution y by the following construction. Given a list 
[ci,C 2 ,...], we write [[ci, c 2 ,... ]] for the sublist formed by omitting repeated entries after their 
initial occurrence. For example, [[1,1, 3,4, 2,4, 3, 2,4]] = [1,3,4, 2]. If the set of distinct elements in 
[ci, c 2 ,... ] is { 1 , 2 ,..., n}, then interpret [[ci, c 2 ,... ]] as the one-line representation of a permutation 
in S n . Now, for y £ Z(S n ), we define a m i n (y) and 0 m m (v) as the permutations in S n given by 

amin(y) = [[ 6 i,ai, 6 2 ,a 2 ,... , 6 /c,a fc ]] _1 and 0 min (y) = [[ai, h, a 2 , b 2 , ■ ■ ■ , a k , b k }}~ 1 

where (ai, b\), (a 2 , 62 ), ■ ■ ■, (ak, bk) are the elements of the set {(a, b) £ [n] x [n] : a < b = w(a)}, 
indexed such that a\ < a 2 < • • • < a^. 

Example 3.2. If y = [4, 7,3,1, 6 , 5, 2] = (1,4)(2, 7)(5, 6 ) then 

«min (y) = [[4, 1 , 7, 2,3, 3, 6 , 5 ]]- 1 = [2,4,5,1, 7, 6 ,3] and 0 min (y) = [1, 3,5, 2, 6 , 7,4], 

In turn, one computes that 
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(a) aimin (g k ) = [2,4,..., 2k, 1,3,... , 2k - 1] and fd m \ n (gk) = [1,3,..., 2k - 1,2,4,... , 2k\. 


(b) a m i n (w 2 k) = [2,4,... ,2k,2k - 1,... ,3,1] and /3 m i n (w 2 k ) = [1,3,... ,2k - 1,2k,... ,4,2}. 

Fix y £ X(Soo) and z £ X FPF (S' 00 ). The following is a corollary of results in [7], and is included 
explicitly in [231 Theorems 6.10 and 6.22], 

Proposition 3.3 (See [23|). The permutations a m i n (y) and /3 m in (z) are the unique lexicographically 
minimal elements of A(y) and ^4 FPF (^), respectively. 

Corollary 3.4. If y (respectively, z) is 321-avoiding, then so is a m in(y) (respectively, 0 m \ n (z)). 

Proof. The set of 321-avoiding permutations is an order ideal under the left weak order (see m 
Proposition 2.4]), and we have u <l tt whenever u £ A(ct,tt) by inspection. □ 


3.2 Diagrams and codes for involutions 

For involutions y £ X{S 00 ), we define 

D(y) = {( i,j ) 6 P x P : j < y(i) and i < y(j) and j < i}, 
Dfpf{v) = {(*, j) € D(y) : j < i}. 


Call these sets involution Rothe diagrams. Observe that D(y) and Dfpf(v) are the subsets of 
positions in the usual Rothe diagram D{y) that are weakly and strictly below the diagonal. Since 
D(y) is invariant under transpose as y 2 = 1, the diagram D(y) uniquely determines D(y). 


Example 3.5. If y = [4, 7,3,1,6, 5, 2] then 


D(y) = 


o o o 
o o o 
o o 


> and D() = < 


o 

o o 
o o 


. 


One similarly computes that 

D{g n ) = {(i,j) ■ 1 < j < i < n} and D(w n ) = {( i,j ) £PxP:i+j<n and j < i}, 


which are the transposes of the shifted shapes of [n,n — 1,... ,2, 1) and (n — 1, n — 3, n — 5,...). 

We discuss a few results that indicate why D(y) and D F pf(z) are the appropriate notions of 
diagrams for involutions. Recall that the cardinality of D(w) is the number of inversions of w £ Soo, 
and so £(w) = \D(w)\. An analogous fact holds for involution Rothe diagrams: 

Proposition 3.6. If y £ T{S n ) and z £ X FPF (52 n ) then £(y) = \D(y)\ and ^ FPF (z) = |H FPF (^)|. 

Proof. When y is an involution, D(y ) is transpose-invariant and the number of diagonal positions 
(i,i) £ D{y) is precisely k(w), so it follows that \D(y)\ = \ (\D(y)\ — n(y)) + n{y) = £(y). If z is a 
fixed-point-free involution, then |Zi> FPF (a:)| = \D(z)\ — k(z ) = £(z) — k(z) = f FPF (z). □ 
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Recall the definition of the code c(w) E Z n from (12.51) . We define the involution codes 
c{y) = (ci(y),c 2 (y), • • • ,Cn(y)) and c FPF (y) = (c FPFa (y), c FPF)2 (y), • ■ ■, c FPF , n (y)) 
for y E T(S n ) as the integer sequences with 

Ci(y) = \{j € [n] : y(j) < i < j and y(i) > y(j)}\, 

c F PF,i(y) = \{j e N : y(o) < i < j and y(*) > vti)} I- 

The i th entries in these sequences count the number of cells in the rows of D{y ) and l) FPF (y), 
respectively. These sequences do not depend in any serious way on n: if y is viewed as belonging 
to a larger symmetric group Sn D S n , then the resulting codes are the same, extended by zeros. 

Example 3.7. If y = [4, 7,3,1,6, 5,2] = (1,4)(2, 7)(5,6) E Z(SV) then 

c(y) = (l,2,2,0,2,l,0) and c FPF (y) = (0,1, 2,0,1,1,0). 

For the involutions g n ,W 2 n E T FPF (52 n ) we have: 

(a) c(g n ) = (1,2,3,... , n, 0,..., 0) and c FPF (y n ) = (0,1, 2,... , n - 1,0,..., 0). 

(b) c(w 2n ) = (1,2,... ,n,... ,2,1,0) and c FPF (u; 2n ) = (0,1, 2,... ,n - 1,...,2,1,0,0). 

Recall the minimal atoms a m \ n {y) E A(w) and /3 m ; n (y) E -4. FPF (y) defined in Section [Til 

Lemma 3.8. If y E X{S n ) then c(y) = c(a min (y)) and c FPF (y) = c(/3 min (y)). 

Proof. Define (a,i,bi) relative to y as before Example 13.21 and fix i E [k], where k is the largest 
descent in y. It is straightforward to check from the definitions that 

c Qi (Anin( 2 /)) = | {j E [i — 1] : bj > afj\ and c bi (/3 min (y)) = \{j E [i - 1] : bj > b*}|. 

On the other hand, since y(t) < t implies y(t) = aj and t = bj for some j. and dj < a,i if and only 

if j < i, it follows that 

CFPF, ai (y) = |{j E [n\ : y(j) < a; < j and y(a») > y(j)}\ 

= | {j E [k\ : dj <di< bj and b t > dj} \ = \{j E [i - 1] : bj > aj}| = c ai (/3 min (y)) 

and likewise 

CFPF,bi(y) = I {j e N : V(j) <bi<j and y(bi) > y(j)}\ 

= |{j E [k\ : dj <bi< bj and a; > dj}\ = \{j E [i - 1] : bj > 6J| = c bi (/3 min (y)). 

As every t E [n] is given by dj or bj for some j E [&], we conclude that c FPF (y) = c(/3 m ; n (y)). The 
proof that c(y) = c(a m ; n (y)) is similar; we omit the details. □ 


One can read off an involution word for y E X(S n ) from its involution code in the following way. 
For any sequence c = (ci, c 2 ,..., c n ) E N n , define 

0(c) = (ci,..., 2,1, c 2 + 1,..., 3, 2, c 3 + 2,..., 4,3,..., c n + n - 1,..., n + 1, n). 


ci terms 


C2 terms 


C3 terms 


c n terms 


In what follows, identify integer sequences (*i,..., ik) with words (s^,..., Sj fc ). 

Proposition 3.9. If y E T(S n ) and z E X FPF (5' 2n ) then 0 (c(y)) E 7 Z(y) and 0 (cp PF (z)) E IZfpf(z). 

Proof. By [ill Remark 2.1.9], if u is any permutation then 0(c(tt)) gives a reduced word for u, so 
this result follows from Lemma 13.81 □ 
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3.3 Involution Schubert polynomials 

Recall the definition of & w for w E Soo from Section 12.31 In this section we turn to the involution 
Schubert polynomials 6 y and (5 FPF defined in the introduction. Recall for y,z E X(5oo) we define 
the involution Schubert polynomial 

& y ,z = ^2 G u e "Poo. 

u£A(y,z) 


We abbreviate by setting & y = &\, y and (5 FPF = & Vn ,z for z E Pfpf (SWO- 

It follows immediately from Theorem-Definition 11.11 that these definitions of & w and 6 FPF are 
consistent with our earlier definitions in the introduction. Write A n for the ring of symmetric 
polynomials in V n = Z[xi ,..., x n ]. For the Grassmannian involution g k given by (|3.1I) we can 
compute & gk and © FPF directly: 

Proposition 3.10. For each k E N it holds that 

6 gk = 2~ k (xi + Xj) E A fc and S FPF = (x* + Xj) E A k . 


Proof. By Proposition 12.131 and the discussion in Section [3TT1 we have <5 gk = s$ k (x i,... ,x k ) and 
G FPF = s§ k (x i,... ,x k )• Checking that these Schur polynomials have the given product formulas is 
a simple exercise from Jacobi’s determinantal definition of sy, see [41L Exercise 1.2.4]. □ 

Involution Schubert polynomials may also be characterized along the lines of Theorem 12.8] as 
follows. We write 6 VtW to the denote the Kronecker delta function, equal to 1 if v = w and 0 
otherwise. Recall the definition of the two-sided weak order <t from Section 12.11 

Theorem 3.11. Fix y E X{S 00 ). Then {&y,z}z&Z(S 00 ) is the unique family of homogeneous poly¬ 
nomials indexed by T(S' OC) ) such that 


& y ,z = S VtZ if y jtr z 


and 


di& y ,z = 


Gj/.ZXlSi 

0 


if Si E Desij(^) 
otherwise 


for all j £ P. 


Proof. We first claim that & y , z has these properties. The polynomials 6 y , z are homogeneous since 
the Schubert polynomial & u is homogeneous of degree i(u). That & VjZ = 5 y , z if y jtx z follows from 
the definition of A(y, z ). The given formula for di& VtZ , finally, is straightforward to check from our 
original definition of & y , z using Proposition 12.41 and Theorem 12.81 

For the uniqueness assertion, suppose {/z}-gx(s OQ ) is another family of homogeneous polynomials 
with the properties of & V}Z described in the theorem. We proceed as in the proof of [321 Theorem 
2.3]. By hypothesis f z = & ytZ = 5 V)Z if y i^T z, so assume that y <t z and that f u = & y , u 
if u E Z{S oa ) is such that £(u) < £(z). Then dif z = di& y ^ z for all i E P, so we deduce that 
f z = 6 VtZ + a y for some a y E Z = P| igP ker5j. Since f z and & VtZ are both homogeneous and since 
6y }Z has degree £(y, z) > 0, the constant a z must be zero so f z = &y )Z as desired. □ 

By induction, we may express & y in terms of divided differences in the following way. 
Corollary 3.12. Let y,z £l(S n ). Then & y . z = d u & ytWn for any u E A(z,w n ). 
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For technical reasons we need a slightly different version of Theorem 13.111 to characterize the 
fixed-point-free involution Schubert polynomials © FPF . Define I FPF as the set of fixed-point-free 
involutions w : P —>• P with the property that, for some sufficiently large positive integer N, it holds 
that w(2i — 1) = 2 i and w(2i) = 2i — 1 for all i > N. Note that each such permutation has infinite 
support. If n is finite and z G I FPF (5 , 2 n ) then let z Q 0 be the infinite product 

Zoo = 2 ■ s 2n+l • S2n+3 ' - s 2n+5 ' ' ' G X FPF . (3-2) 

We also let 1 FPF = l^ G Z fpf be the map P — > P with 2i — 1 i— 2i and 2 i i—>■ 2i — 1 for all i. Although 
no elements of X FPF belong to Soo , we define w xi Sj for w G Z FPF exactly as if w were in and we 
define the (now infinite) set Des r(w) again by (12.411 . 

If ^ G T FPF (S^n), so that zs 2 n+i G ^FPF(*S , 2 n+ 2 ), then it follows either as a straightforward exercise 
or from the more general statement [23) Lemma 3.2] that A(v n , z) = A(v n+ i, zs 2 n +i) and therefore 
(5 FPF = 6 FPF n+1 . By induction, if y G Ip PF (<S 2 m ) and z G T FPF (52n), then S FPF = © FPF whenever 
Voo = z oo- It is therefore well-defined to set 

S F ^ F = 6 fpf = & Vn ,z for any ^ G X FPF (5 2n ) and n G P. 

Since X FPF = Unepi^ 00 : z ^ ^FPF(<52n)}> this defines 6 FPF for every w G X FPF . 

Corollary 3.13. It holds that {(5 FPF }. e j FpF is the unique family of homogeneous polynomials 
indexed by Z FPF such that 


6 fpf = 1 
J-FPF 


and 


< 9 ,;( 5 fpf 


© 

0 


FPF 

zy\Si 


if Si G DesR(z) and zxsjG X FPF 
otherwise 


for all i G P. 


Proof. Note that if z G I(S- 2 n ) is not fixed-point-free then it cannot hold that v n <t z (since 
y <t z implies that y has at least as many fixed points as z) so & Vn ,z = 0. From this observation, 
the assertion that the polynomials {® 2 PF }rgi FPF have the given properties is an easy exercise from 
Theorem 13.111 The uniqueness assertion follows by the same argument as the one in the proof of 
Theorem 13.111 mutatis mutandis , after replacing the involution length £ which is not well-defined 
on Z FPF by the function f? FPF : X FPF —> N given by £ FPF (zoo) = £(v n , z) for z G I FPF (<S l 2 n ). □ 

Given a sequence i = (h,i 2 , •••,*«) G N n , we let x l = ■ ■ ■ x 1 ™ and write x l <i ex xi when 

i <i ex j G N n , where <i ex denotes the lexicographic order on sequences. A different convention is 
sometimes taken to define lexicographic order on monomials, as we explain in the following remark. 

Remark. Viewing x l as a word in the alphabet {xi,X 2 , ■ • • } defines a sequence 

T(aP) = (l,.„,l,2,.„,2,...,n,...,n). 

i\ times 12 times i n times 


For example, T (a^ 1,1,1 )) = T(xiX 2 X 3 ) = (1,2,3) and 'L (x^ 0,2,1 ^ 1 ) = T (x^x^) = (2,2,3). One checks 
that on the set of monomials of any fixed degree, the map x l i —> ^>{x l ) reverses lexicographic order; 
e.g., we have X 2 X 3 <i ex x 1 .T 2 .r 3 but (1,2,3) <i ex (2,2,3). For this reason, the order <i ex that we 
have defined on monomials is sometimes (e.g., in [2j) referred to as the reverse lexicographic order, 
though for us this order is the usual lexicographic total order. 
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We now show how to read the involution codes c(y ) and Cfpf(z) from the polynomials 6 y and 
© FPF . As these codes determine y £ Z(S'oo) and 2 ; € 2fpf(S , oo) ! one can reconstruct the index of an 
involution Schubert polynomial from the polynomial itself. 

Proposition 3.14. Let y € X(S' 00 ) and 2 £ Tfpf(S'oo)- The lexicographically least monomials in 
the involution Schubert polynomials 6 y and © FPF are and x C7FF ^ respectively. 

Proof. One checks that if u, v! £ S 00 with u <i ex v! (interpreting u and v! in one-line notation), 
then c(u ) <i ex c(u'). It therefore suffices by Proposition 13.31 and Lemma 13.81 to show that x c is 
the lexicographically least monomial in <5 U . This property is clear from the proof of [2J Corollary 
3.9], on noting that x c ^ = xx> hot ( u ) i n the notation of [2, §3], and that <i ex is what the authors of 
[2J call the reverse lexicographic order on monomials. □ 

3.4 Cohomology of flag varieties revisited 

Recall the notation of Section 12.41 Throughout this section, we let (5 be a non-degenerate bilinear 
form on C n which is symmetric or skew-symmetric, and define K C GL n (C) as the subgroup of 
matrices preserving (3. The group K is given by the orthogonal group O(n) when /3 is symmetric, 
and by the symplectic group Sp(?r) when /3 is skew-symmetric (which can only occur if n is even). 

As explained in [57] §10], the orbits of the symmetric subgroups O(n) and Sp(n) on Fl(n) are 
naturally indexed by Z(S n ) and l FPF (n), respectively. To refer to these indexing sets, we define 

To(n) = Z(S n ) and X Sp ( n ) = X F PF(<S'n)- 

The corresponding il-orbits may then be described by rank conditions analogous to the ones (|2.8D 
for Schubert varieties. Explicitly, we may define a FT-orbit associated to an involution y £ Z F by 

Yy = {F. : rank (/3| Fi x Fj ) = rk y {i,j) for each i.j £ [n]} , 

where f3\ FiXFj denotes the linear map F, —» F* given by v >-)■ f3(v,-), and rk y (i,j) is as in (12.71) . 
It is not hard to see that is A'-stable, and that Fl(n) is the disjoint union [_} y& x K Yy Wyser 
discusses why Y^ is actually a single Ji-orbit in [54l §2.1.2], 

Let Y y K denote the Zariski closure of Y y as in Section l2~4l Then Y y x is again defined by rank 
conditions; namely (see joT . Proposition 2.4]) 

Y y = {F. : rank (/3\ FiXFj ) < rk y (i,j) for each i,j £ [n]} . (3.3) 

Recall from (12.81) that the Schubert variety X w for w £ S n is the set of complete flags F, . such that 
dim (proj Fj —> Vj) < rk w (i,j) for each (i,j) £ [n] x [n]. Fulton shows in [15] that a proper subset 
of these rank conditions actually imply all of the rest. Specifically, to determine X w one only needs 
the conditions corresponding to pairs (i,j) in the essential set of the Rothe diagram D{w). In 
general, the essential set of a diagram D C P x P is 

Ess(-D) = {(i,j) £ D : (i + 1, j), (■ i,j + 1), (i + l,j + l)<£ D}. 

Observe that Ess(F) is the set of southeast corners of the connected components of D. 

Example 3.15. If y = [4,7,3,1,6, 5,2] as in Example 13.51 then 

Ess (D(y)) = {(2, 3), (2, 6), (3,2), (5,6), (6, 2)} and Ess (D(y)) = {(2, 3), (2, 6), (5, 6)}. 
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The rank conditions (13.31) giving admit the following analogous simplification. 
Proposition 3.16. The K -orbit closure associated to an involution y £ Xk satisfies 
Y y h = {i 7 . : rank ((3\ F . xFj ) < rk y (i,j) for each (i,j) £ Ess(D)} 
where D = D(y ) when K = O(n) and D = D FPF (y) when K = Sp(n). 

Proof. Fix y £ Xk and let C tJ denote the set of complete flags F, £ Fl(ra) satisfying the condition 
rank (p\FixFj) < rk y (i,j). Since y is an involution, we have C l3 = Cji , so (13.31) implies that Yff is 
the intersection of the sets Cjj for 1 < j < i < n. Define the implication graph of y as the directed 
graph on {(i,j) : 1 < j < * < n} with an edge from (i, j) to (k, l) if the two cells are adjacent in 
the same row or column and C\j C Cki- 

First assume K = O(n) so that D = D{y). If (i,j) £ D, then rk y (i — 1 ,j) = rk y (i,j — 1) = 
rk y (i,j) so Cij is contained in Ci-ij and Cij -If (i,j) £ D, then either j > y(i ) or i < y ~ 1 (j). In 
the first case, rk J/ (i — l,j) = iky(i,j) — l, so ij C C^; in the second case, we deduce i C C\j 
by an analogous argument. Thus, each cell in D is the source of edges in the implication graph 
going both north and west, while each cell not in D is the target of an edge going either south or 
east. It follows that every (i. j) £ D can be reached by a directed path in the implication graph 
starting at some cell in Ess(D), and every (i,j) D can be reached by a directed path starting 
at either (1,1) or a cell in D. Since (1,1) ^ D only if ic(l) = 1 in which case Ci i = Fl(n), we 
conclude that each Cij contains C pq for some (p. q) £ Ess(D), so l” y A is the intersection of the sets 
C pq for (p,q) £ Ess(.D), as desired. 

Now suppose K = Sp(n) so that D = D FPF (y). Since in this case (3 is skew-symmetric and 
y is fixed-point-free, the numbers rank (/3|_F iX Fi) anc ^ r ky(b*) are always even and bounded above 
by i. Hence C \ t i = Fl(n). We claim, moreover, that Cij-\ C Ca for all 1 < i < n. To show 
this, suppose F. £ Cij- 1 . If rank = rank (/3|_p iXFi ) or rk y (i,i - 1) < rk y (i,i) then 

clearly F, . £ C % j. otherwise, it must happen that rank (^FixFi-j) = rank(/3| FixFj) — 1 is odd and 
rk y (i,i — 1) = rk y (i,i) is even, so the strict inequality rank {(3\F i xF i - 1 ) < rk y (i,i — 1) holds, which 
again implies F, £ Ca, From the claim just shown and (13.31) . we deduce that F A is the intersection 
of Cij for 1 < j < i < n. The proposition thus follows as in the orthogonal case, by considering the 
implication graph on the set of cells {(i,j) : 1 < j < i < n} strictly below the diagonal. □ 

As for Schubert varieties, to each orbit closure Yff' there is an associated cohomology class 

[Yy K ] £ H* (Fl(n), Z) for y £ I K . (3.4) 

The Borel isomorphism (12.91) identifies these cohomology classes with elements of the quotient 
V n /(Aif). Combining [Ml Propositions 2.1 and 2.7] with the discussion in [55], §1.3] gives the 
following analogue of (12.101) describing the action of the divided difference operators on [F A ]. 

Theorem 3.17 (Wyser and Yong [541155] ). Let i € [n — 1] and y £ Xk and set s = s*. Then 



' [Yy^s] if y xi s = sys < y 
( 2 K/Ll if y x s = ys < y and ys £ X K 
^0 if s £ Des R (y) or y xi s X K . 
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Remark. We include a sketch of the proof of this theorem since it does not immediately follow 
from Wyser and Yong’s results in [5T 55] that di [Y y A ] = 0 in the case when y xi s Ik- Note that 
this occurs only if K = Sp(n) and y x s = ys so that ys is not fixed-point-free. 

Proof sketch. Fix i £ [n]. Suppose X is a subvariety of Fl(n), and consider the set (variety, in fact) 
X' obtained by replacing each flag F, £ X by all flags E. with Ej = Fj for j ^ i (informally, 
remove any conditions defining X that restrict Ff). If dirnY' = dimY + 1, then there will be 
an integer d such that, generically, each flag E . £ X' arises from d distinct flags in X, in which 
case di[X] = d\X'\. If, alternatively, dimY' 7 ^ dimY + 1, then <9,;[X] = 0. For a more detailed 
justification of these assertions, see na Chapter 10]. 

One can use the rank conditions on Y y A to understand (Y^)' and the integers d. For example, 
suppose K = Sp(n) and let y £ Ik such that y xi Sj is not fixed-point-free. Then y(i) = i + 1. 
We claim that row i and column i of Ess (D FPF (y)) are both empty. Indeed, row i of the involution 
Rothe diagram D FPF (y) is empty by definition, while if (k, i ) € -Dfpf^) for some k < i , then we also 
have (k + l,i) £ D FPF (y) (for otherwise y(k) = i + 1), in which case (k,i) ^ Ess(-D FPF (y)). Thus, 
any of the rank conditions defining Y y A in Proposition 13.161 that involve F t are implied by others 
that do not, so (Y y A )' = T y A and therefore d j[Y A ] = 0. The other cases follow similarly. □ 

The theorem shows that one may compute polynomial representatives for the cohomology classes 
(13.41) just as for Schubert classes, i.e., by applying divided difference operators to suitable repre¬ 
sentatives for the longest element w n £ S n . Wyser and Yong identify such representatives in [ 55] . 
To state their result, recall from the introduction that 

T Si n) = II ( x i + x i) and T «?2 ( „ n) = II ( x i + x i )■ 

l<i+j><n l<i+j<2n 

i<j i<j 

Wyser and Yong prove the following as [55], Theorem 1.1]: 

Theorem 3.18 (Wyser and Yong [55j). Let y £ Ik and choose any u,v £ A(y,w n ). Then 

Tl = [T A ] and d v T* n = d v T^. 

Let y £ Ik and u £ A(y,w n ). If K = O(n) then define = 2 K ^ y ' ) ~ K ( w Ad u '^ ( ^^ a nd 

if K = Sp(n) then define T= d u Tf^} n \ Theorems 13.171 and 13.181 show that T y is then a 
representative of [Y^]] moreover, these polynomials do not depend on the choice of u, and so 
are unambiguously indexed by Ik C S n . Wyser and Yong note in [55, Theorem 1.1] that these 
representatives are nonnegative integer linear combinations of ordinary Schubert polynomials. We 
can identify this decomposition explicitly by showing that Wyser and Yong’s polynomials are 
actually scalar multiples of the involution Schubert polynomials defined in the previous section. 

Theorem 3.19. Let y £ X(S n ) and z £ I FPF (S 2 n )- Choose u £ A(y,w n ) and v £ A(z,W 2 n )- Then 

e y = 2~^d u T^ and ©f 1F = d v T^ 2n \ 

Remark. In the following proof, we need to suppress n, so we define T y = 2~ K ^T^ n ' > for y £ 
X(S n ) and T^ F = for z £ T FPF (S' 2 n ), with denoting the permutation of P defined from 

z by (13.21) . This gives us a family of polynomials T y indexed by elements of X(S' 00 ), and a family 
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of polynomials Y FPF indexed by elements of the set X FPF defined in the previous section. The fact 
that the polynomials Y y and Y FPF given in this way are well-defined, independent of our choices of 
n, is proved by Wyser and Yong as [55j Theorem 1.4]. 

Proof of Theorem \3.19l Note that T y = 2A n / 2 \d v T < ^ff > and T FPF = d v T^ffj 2n) for any u £ A(y, w n ) 
and v £ A(z, w 2n ) by Theorem l3.18l It therefore suffices to argue that {'^y} y ei(s 00 ) an d {Y FPF } ze jr 
have the properties in Theorem 13.111 and Corollary 13.131 that uniquely characterize {&y}y£i(s 00 ) 
and {<S^ PF } _ e x FPF - For this, we first claim for all y £ Z(S 00 ) and z £ X FPF and i,j £ P that 

d T _ j'^'yxisi if i £ Desfi(y) ^. t fpf _ f TFPF ,- if .7 e Des/^z) and z xi s t £ X FPF 

lo otherwise J 1 0 otherwise. 

Choose n £ P such that y £ I(S n ) and z = Woo for some w £ X FPF (S 2 n)- Then T y £ V n -i and 
Y FPF £ Pan- 1 , so the claim holds automatically when i > n and j > 2n since both sides of the two 
equations are zero. When s* £ Des n(y) and Sj £ Desij(^) and z xi Sj £ X FPF , the desired identities 
follow directly from the definitions and Lemma [2. II Finally, suppose i £ [n — 1] is not a descent of y 
and j £ [277. — 1] is not a descent of 2 . Then by the preceding case T y = diT yyiSi and T FPF = dj Y FPF S 
so as the divided difference operators square to zero, it follows that djT y = d,Y FPF = 0. 

It remains to show that Ti = T FP ^, = 1 and that if z x s ^ Z FPF then d % T FPF = 0. Recall from 
the discussion above that 2 K ^T y = \Y^ n \ if y £ l(S n ) and Y FPF = [yJ p ^ 2 " ’] if z = w 00 where w £ 
X FPF (S 2n ). The rank conditions in Proposition 13.161 show that Y | ° 1 '"' 1 = Fl(n) and Yiff l(2n> = Fl(2n), 
so these orbit closures correspond to the identity elements in their corresponding cohomology rings. 
As the Borel isomorphism (|2.9I) is an isomorphism of rings and since T y and Y FPF are evidently 
homogeneous polynomials, it is immediate that T p = T^ (,,) = 1 and T FP p F = Y^ 2 ”'* = 1. 

Finally suppose z £ X FPF and j £ [2 n — 1] are such that z xi Sj X FPF . As before, we then 

have ^ = Woo for some w £ X FPF ( 52 n ), and necessarily j £ Des r(w) and w x Sj = wsj ^ X FPF ( 52 n )- 
Theorem 13.171 implies dj [Yi; P( 2?< ' ) ] = 0, so as Y FPF = [Y« p< ' 2ri ^] it follows that <9,'Y FPF = 0. We 
conclude by Theorem 13.111 and Corollary 13.131 that Y y = 6 y and Y FPF = © FPF for all y £ I(Soo) 
and z £ X FPF , and so our result follows from Theorem 13.181 □ 

3.5 Product formulas 

Theorem 13.191 establishes an explicit product formula for & Wn and and in this section we 

generalize that result to the following class of involutions: 

Definition 3.20. An involution y £ Z(S 0G ) is weakly dominant if it has the form 

y = ( 1 , 6 i)( 2 , 6 2 ) • • • (Mfc) 

for some k £ N and some integers b ±, 6 2 , • • •, with > k for all i £ [k]. 

If y = (1, 6 i)( 2 , 62 ) • • ■ (k, bf.) £ X(5 n ) is weakly dominant then we define r(y) £ S n _k as the 

permutation given in one-line notation by 

r(y ) = [h-k,b 2 -k,...,b k - A:, ci, C 2 ,... ,c n _ 2 fc] 

where c F < c 2 < ■ ■ ■ < c n _ 2 fc are the elements of { 1 , 2 ,..., n — k} \ {61 — k,b 2 — k ,..., b k — k}. 
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Example 3.21. We have r ((1, 6)(2, 5)(3, 8)) = [3,2,5,1,4] and r(g^) = 1 and r(wk ) = w\k/ 2 ]- 
More generally, if y = u~ l gkU for any u £ Sk, then y is weakly dominant with r(y ) = u. 

It follows from Section \2 . 21 that the permutation r(y) has these basic properties: 

Observation 3.22. If y £ 1(S n ) is weakly dominant with k = n{y) distinct 2-cycles, then r{y) 
belongs to S n _k with largest descent at most k , so D(r(y)) C [A;] x [n — k\. 

Recall that D{gk) is the transpose of the shifted shape of 5k+ 1 - For any permutation u, define 
Ek(u) = {{j + k,i ) : (i,j) £ D(u)} as the transpose of D(u), shifted down by k rows. 

Lemma 3.23. Suppose y £ Z(Soo) is weakly dominant and k = K(y). Then 

D(y) = D(g k ) U E k {r{y )) and D FFF (y) = D FFF (g k ) U E k (r(y)). 


Example 3.24. Before commencing the proof of this lemma, it is helpful to give an example 
illustrating the result. Let y = (1, 6 )(2, 5) (3, 8 ) = [ 6 , 5, 8 ,4, 2,1, 7, 3] as in Example 13.211 Then 


D(y) 


O 

o o 
o o o 

< O o o , 

o 



D(r(y )) 



o 


and the lemma’s claim that D(y) = D(g$) U E^{r{y)) is evident. 

Proof of Lemma, \S. 23 1 By definition, y = (1, &i)(2, 62 ) • • • (fc, bk) for distinct integers bi > k. It is 
clear by construction that D(g k ) C D(y)- Fix (i,j) € P x P. We claim that (j + k,i) € D(y) if and 
only if (1, j) £ D(r(y)). Recall that (j + k,i) € D(y) if and only if 


j + k < y(i) and i < y(j + k ). 


(3.5) 


The claim holds when i £ [k] since then the first of these conditions is equivalent to j < r(y)(i), 
while the second may be rewritten as 

i<y{j + k) j + k £ { 61 , 62 , ■■■A} j(£r(y)(\i}) <s> i<r(y)~ l (j). 

On the other hand, if i > k then (i,j) (j D{r{y)) by Observation 13.221 and (j + k,i ) ^ D(y) since 
one checks that the conditions (13.511 never simultaneously hold. Hence the set of positions in D(y) 
below the k th row is precisely Ek(r(y)). The latter set is contained entirely below the diagonal 
since D(r(y )) C [k] x [n — k\, so the lemma follows. □ 

Recall the definition of dominant from before Proposition 12.13l We see by the following propo¬ 
sition that every dominant involution is weakly dominant. 

Proposition 3.25. Let y €X(5' 00 ). The following are equivalent: 

(a) y is dominant. 
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(b) D(y) is the transpose of a shifted shape. 

(c) y is weakly dominant and r(y) is dominant. 

Proof. The equivalence of (a) and (b) is immediate from the definitions of dominant permutations 
and involution Rothe diagrams. By Lemma 13.231 it is clear that if y is weakly dominant then the 
transpose of D(y) is a shifted shape if and only if D(r(y )) is the diagram of a partition, i.e., r{y) is 
dominant. It is a straightforward exercise to check that a 132-avoiding (i.e., dominant) involution 
is weakly dominant, so it follows that (a) and (c) are equivalent. □ 

We now prove Theorem 11.31 which is the involution analogue of Proposition 12.13T a). 

Theorem 3.26. Suppose y £ X{S 00 ) and z £ I FPF (S' 0O ) are dominant. Then 

& y = 2~^ yS> JJ (Xi + Xj ) and 6 FPF = (Xi + Xj ). 

(i,j)£D(y) RDe-tWz) 

Proof. We prove the first identity, since the formula for (3 FPF follows by essentially the same ar¬ 
gument. To begin, it is helpful to note (see (JT, §2.1]) that the Rothe diagram of a permutation 
w is the complement in P x P of the hooks through the points ( i,w(i )) for i £ P, where the hook 
through a cell (i, j) is the set of positions of the form (i + t,j) or ( i,j + t ) for t £ N. It follows 
that if w is dominant, so that D(w ) is a partition, then the northwest corners of the complement 
of D(w) are all of the form ( i,w(i )), and that if (i,w(i)) is such a corner then i is a descent of w if 
and only if the (i + l) th row of D(w ) is shorter than the i t,h row. 

Let y £ I(S n ) be a dominant involution. The desired formula holds when y = w n by Theorem 
I3.19[ so assume y < w n and that the product formula is valid for all dominant involutions z £ X(S n ) 
with £(y) < l(z). The Rothe diagram of D(y) is strictly contained in D(w n ) = (n — 1, n —2,..., 2,1), 
so we may define j £ [n] to be minimal such that (n — j. j) D(y), and then define i £ [n — j] to 
be minimal such that (i,j) fi D{y). The cell if. j) is then a northwest corner of the complement of 
D(y), so j = y(i). Moreover, rows i, i + 1,..., n — j + 1 of D(y) all have length j — 1, so we must 
have Si ^ Des n(y) and also j < i, since D(y) is symmetric under transpose. 

Using these facts and the interpretation of the Rothe diagram as the complement of the hooks 
through the points of a permutation, it is a straightforward exercise to check that 

D{y » Si) = D(y)U {(i,j)}. (3.6) 

We omit the details, since the argument is easier to visualize than to transcribe and is similar to 
the proof of nn Proposition 2.6.7]. By Proposition 13.251 the identity ([3.611 implies that y x Si is 
itself a dominant involution of greater length than y, so by induction and Theorem 13.Ill we obtain 


— di&v'As; — 


2~^y™ Si \xi + 


■ j) IJ (Xk + Xl) 
(k,l)&D(y) 


(3.7) 


The product Tl(k i)eD( y )( Xk + x i) i s Si-invariant since the map (k,l) i->- (sj(fc), Si(l)) preserves the 

transposed shifted shape D(y), which has the same number of cells in rows i and i + 1 and no cells 
in columns i and i + 1. Since di(fg) = fdi(g) when Sif = /, equation (13.71) becomes 

6 y = 2~ K ^y' As i> JJ [x k + xi) ■ di(xi + xj). 

(k,l)£D(y) 
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This transforms to the desired formula for & y on checking that + Xj) is 1 if i ^ j and 2 
otherwise, and then noting, since n(a) is the number of diagonal cells in D(a) for any a G Z(S'oo), 
that n(y x Si ) — n(y) is likewise 0 if i ^ j and 1 otherwise. □ 

For p, q G N define <h P)(? as the map Voo{x\ y) —> V p + q (x) with 

$P, q ■ f{x; y) H- f(x i,x 2 , ...,x p , 0,0,; -x p+ 1 , -x p+ 2 , • • ■, - x p+q , 0 , 0 ,...). 

In other words, & Ptq is the ring homomorphism which maps Xi i —> Xi and yj H > — x p+ y for i € \p\ 
and j € [q] , while mapping all other variables to zero. Suppose 2 ; G I(S n ) is weakly dominant and 
k = k(z). If Ek(u) is defined as before Lemma 13.231 then it follows from Observation 13.221 that 


11 (xi + xj)= (xi + x j+k ) = JJ (x t - y 3 ) J . (3.8) 

This fact leads to the following result, generalizing the previous theorem. 

Theorem 3.27. Suppose z G T{S 00 ) is a weakly dominant involution. Let p = k(z), define n as 
the smallest integer such that z G S n , and set q = n — p. Then 

&Z = & 9p ■ $p,q {&r(z)(x] y)) and <3 FPF = 6 FPF • ( 6 r ( 2 )(x; y)) 

where the second identity applies only in the case when z is fixed-point-free. 

The discussion in Section 13.41 makes it natural try to interpret this statement geometrically, and 
we outline a geometric proof of the theorem along these lines in Appendix lAl 


Proof. Since z is weakly dominant we may write z = (1, &i)( 2 , 62 ) • • • (p, b p ) for distinct integers 6 * all 
greater than p. First suppose b\ > 62 > ■ ■ ■ > b p . One checks that z is then dominant, so it follows 
from Proposition 12. 13f a) that the right most expression in (13.81) is precisely $ P , q [f3 r ( w \{x\ y)j . By 
Proposition 13.101 noting Example 13.51 it always holds that & gp = IT;, j)^b(g )( x i + Xj), so the 

desired formula for & w follows by Lemma 13.231 and Theorem 13.261 

Suppose alternatively that there exists an index i G \p — 1] such that b{ < bi +1 . Then i is a 
(right) ascent of both z and r(z), and evidently z x Sj = SiZSi is also weakly dominant, so we may 
assume by induction that & z -x 8i = 6 gp ■ Tp.g (& r (z><\si)(x',y)) ■ As & gp G A p by Proposition 13.101 it 
follows by our inductive hypothesis and Theorem 13.111 that 


— di&zxsj — 


® g p ‘ *&p,q (&r(zxsi)(x', ?/)) — ■ di^ p q (&r(zxsi)(x, ?/)) . 


Since r(z x sf) = r(z)sj > r(w), and since di acts only on the Xi variables when applied to an 
element of 7 > 0 o (x\y), we have di$ P;q (& r (zxsi)( x ',y)) = &p,q {&r(z)( x ',y)) • Substituting this into the 
preceding equation gives the desired formula for 6 Z . When z is fixed-point-free, the analogous 
identity for (5 FPF follows by a similar argument. □ 
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3.6 Involution Stanley symmetric functions 

For y,z £ X(S' 00 ), we recall from the introduction the involution Stanley symmetric function 

F y ,z = Fu G A ‘ 

uGA(y,z) 

We abbreviate as usual by setting F y = F\ y and F fpf = Fv n , z for z £ 2i FPF (52 n )- 

Observe that F y _ z = 0 if y ji t z , with <r as in Section 12.11 The following slight modification 
to Theorem 12.151 holds by Theorem-Definition 11.11 

Observation 3.28. If y, z £ l(Soo) then F V)Z = f3 y , z ,\S\ where the sum is over partitions A 

and is the number of strict tableaux T of shape A with rrw(T) £ TZ(y,z). 

Note that /3 y , z ,\ = 0 if A is not a partition of g(y,z), so the sum appearing in the observation’s 
formula for F y z is hnite. From Theorem 12.161 we obtain the following corollary: 

Corollary 3.29. If y,z €Z(S’ 00 ) then \TZ(y,z)\ = J2\Py,z,\f X - 

Thus, to count the number of elements in the sets 7 Z(y,z), we need only determine the Schur 
decomposition of the symmetric functions F yz . The rest of this section is spent proving a few facts 
about such decompositions that follow directly from properties of Stanley symmetric functions and 
involution words. 

Just as for ordinary Stanley symmetric functions, F y and F fpf are skew Schur functions when 
indexed by 321-avoiding permutations. In detail, given a sequence of nonnegative integers c = 
(ci, C 2 ,..., c n ) whose nonzero entries occur in positions k\ < ■ ■ ■ < ki, let skew(c) be the set of cells 
(i.j) £ P X P, with 1 < i < l, such that 

i — ki — <j + L<i — ki where L = l — ki — Ck r 

By [3L §2-2.2], it follows that if c = c(w) is the code of a 321-avoiding permutation w, then skew(c) 
is a skew shape. Recall that S n = (n — 1, n — 2,..., 2,1). 

Example 3.30. If y = v n £ S 2 n then c{y) = (1,0,1,0,... ,1,0) and skew(c(y)) = 5 n+ i/5 n . 

If y £ Sqo is 321-avoiding then Proposition l2.17l asserts that F y = s s k ew ( c ). The following parallel 
statement holds for involution Stanley symmetric functions: 

Proposition 3.31. Suppose y £ Z(S' 00 ) is 321-avoiding. Then skew(c(y)) = X/y is a skew shape 
and F y = s^/^. If y is fixed-point-free, then skew(c FPF (y)) = 'y/v is a skew shape and F y PF = s^/ u . 

Proof. By Theorem 13.11 and Proposition 13.31 A(y) = {omin(y)} and (in the fixed-point-free case) 
Al FPF (y) = {/?min(?/)}- By Corollary 13.41 a m i n (y) and /3 m in(y) are themselves 321-avoiding, so the 
result follows from its analogue for ordinary Stanley symmetric functions by Lemma 13.81 □ 

We next classify the involutions y £ S oo for which F y and F fpf are Schur functions. To begin, 
we note the following lemma which derives from the discussion after [12], Proposition 5.4], 

Lemma 3.32 (Eriksson and Linusson [12]). A permutation w £ S ^ is both 321-avoiding and 
2143-avoiding if and only if either w or w~ 1 is Grassmannian. 
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If w G Sk and m G P then we define the shifted permutation 

l m x w = [1,2,..., m, w(l) + m, w(2) + m,... w(k) + m] G 5fc +m . (3-9) 

For any v G S m we similarly define (with slight abuse of notation) v x w = v ■ (l m x w) G Sk+ m - 
With this convention, if z G is a fixed-point-free involution, then v m x z x v n is as well. 

Example 3.33. We have I 4 x W 4 = (5, 8 )( 6 , 7) and V 2 x W 4 x V 5 = (1, 2)(3, 4)(5, 8 )( 6 , 7)(9,10). 

Recall that we define gk = (1, k + 1 )(2, k + 2),... (&;, 2k) G S- 2 k- 

Proposition 3.34. If y G S 00 is a Grassmannian involution then y = l m X gk for some m,k € N. 

Proof. Let y G X(S' 00 ) be Grassmannian. It suffices to show that if k = y{ 1) — 1 is positive then 
y = gk- For this, observe that the Rothe diagram D(y) contains the cells (1 ,i) for all i G [k], 
and therefore also the cells (z, 1) for i G [k] since D{y) = D(y~ 1 ) = D{y) T . As k is evidently 
the unique descent of y, D(y) has no cells below the k th row or (by symmetry) to the right of 
the k th column; hence D(y) C [k] x [k]. On the other hand, by the definition of a Grassmannian 
permutation before Proposition 12.131 it holds that each nonempty row in D{y) contains at least as 
many cells as the row above it. Since the first row of D{y) already has k cells, it follows that in 
fact D(y) = [k] x [k] = D(gk), so y = gk as desired. □ 

The permutations y G I(Soo) whose involution Stanley symmetric functions are single Schur 
functions turn out to have a very restricted form, which we now describe. 

Proposition 3.35. Suppose y = l m x gk-i and z = v m x gk x v n for some m,n,k. Then 

F y = Pf F = and |77(y)| = |77 FPF (^)| = f S F 

Proof. These identities follow directly from Proposition 13.311 and Corollary 13.291 □ 

Theorem 3.36. Let y G X(S'oo) and z G X F p F (5oo). 

(a) F y is a Schur function if and only if y = l m x gk for some m,k G N. 

(b) Ff PF is a Schur function if and only if z = v m x gk x v n for some m, n G N and k G P. 

Proof. Suppose F y is a Schur function. Combining Theorems 12.181 and 13.11 with Proposition 13.31 
and its corollary shows that y must be 321-avoiding and that a m i n (y) must be both 321-avoiding 
and 2143-avoiding. By Lemma 13.321 either a m [ n (y) or its inverse is therefore Grassmannian. It is 
apparent from the definition that a m i n (y ) -1 is Grassmannian only if y is the identity or a simple 
transposition (as the only 321-avoiding transpositions are simple), in which case cu min (y) is equal 
to its inverse. We conclude therefore that a m i n (y) must be Grassmannian. We now argue that y 
must be some shift of gk- To this end, suppose ( 0 , 6 ) and ( a',b') are pairs of positive integers with 

a < b = y(a) and a' < b' = y(a'). One checks that it cannot occur that a < a' < b' < b (as then y 

would contain the pattern 321) or that a < b < a' < b' (as then a m in(?/) would contain the pattern 
2143), so if (a, 6 ) and ( a',b') are distinct with a < a' then a < a' <b <b'. Using this property, it 
is straightforward to deduce that y must have the form l m x gk for some rn, k G N. We leave this 
exercise to the reader. 
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Suppose next that F fff is a Schur function. By the same set of results as cited in the previous 
paragraph, it follows that z must be 321-avoiding and either (z) or its inverse must be Grass- 
mannian. Let (a,, bi) for i = 1,2,3,4 be elements of the set {( i,j ) £ P x P : i < j = One 

checks that it cannot occur that a\ < < 62 < &i (as then z would contain the pattern 321) or 
that a\ < <32 < bi < a% < 62 < ^3 (as then (3 m i n (z) and its inverse would both contain the pattern 
132546) or that a± < 02 < b\ < 62 < «3 < «4 < &3 < b 4 (as then /3 m i n (z) and its inverse would 
both contain the pattern 13245768). Using these properties, it is a similarly elementary exercise to 
deduce that 2 must have the form v m x x v n for some m,n,k £ N. We again leave this to the 
reader. 

This proves one half of the theorem, and the converse holds by Proposition 13.351 □ 

A permutation is antivexillary if it is both 321-avoiding and 351624-avoiding. For an explanation 
of this terminology, see [121 Proposition 5.1]. 

Corollary 3.37. Let y £ X(S' 00 ) and z £ X FPF (S , 2 n ). 

(a) F y is a Schur function if and only if y is vexillary and 321-avoiding. 

(b) Xj PF is a Schur function if and only if z is antivexillary and 231564-avoiding. 

Proof. Given the theorem, part (a) is immediate from Lemma 13.321 and Proposition 13.341 To prove 
part (b), one must show that a fixed-point-free involution z is antivexillary and 231564-avoiding if 
and only if z = v m x x v n for some m, k € N. We leave this exercise to the reader. □ 

3.7 Stabilization 

To prove stronger statements about involution Stanley symmetric functions, we must leverage the 
results in Section 13.51 we discuss methods for this here. If / is a power series in the variables 
x = {x\, X 2 , • • • }, then we write r n (f) or f(x 1 ,..., x n ) for the power series formed by setting the 
variables x n + 1 = x n +2 = ■ ■ ■ = 0. Observe that if / £ Voo then r n (f ) £ V n and that if / £ A then 
r n(f ) € A n , where A n denotes the ring of polynomials in V n fixed by the action of S n . 

Following [40], we define the stabilization (in degree n £ NJ of & w for w £ S oo as 

stb n ( 6 ™) = F w (x 1 , ...,x n )€ A n . (3.10) 

By Proposition 12.121 this formula extends by linearity to a map stb n : Voo A n . We then have 

stb n(&y,z) = Fy,z(x i, ..., x n ) for y, Z £ l{Soo) 

and lim n _ > , 00 stbn)©^) = F y ,z, where the limit is interpreted in the sense of formal power series, 
with a sequence of power series defined to be convergent if the sequence of coefficients of any fixed 
monomial is eventually constant. 

By applying stb n to both sides of the identities in Theorem 13.271 one might hope to show that 
similar factorizations hold for F y and Ff PF . This strategy cannot work in general, since stabilization 
is not a ring homomorphism and may fail to preserve products. However, we will find that in certain 
cases of interest the maps stb n do behave as we would wish, but to prove this we will require several 
preliminaries about these operations. For w £ Sk, recall from (13.91) the definition of l m x w € S m+ k 
and note the following corollary of (11.31) . which appears as m Proposition 2.8.1]: 
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Corollary 3.38. If w £ Soo then stb n (@„,) = &i n xw(xi,X 2 , ■ ■ ■, x n ) for all N > n. 

More usefully, we can express stb„, in terms of certain modified divided differences. Following 
[40], we define the isobaric divided difference operator 7Tj : Voo ~> Poo for i £ P by 7 r ? ;/ = di(xif). 
For example, iTi(xiXi + i) = di(x?Xi+i) = XiXi + \. One checks that n'f = 7Tj and that 

7 r i(fg) = f-'Kig whenever /,g £ Poo such that Sif = f. (3-11) 

In particular 7 r*(/) = / • 7 Tj(l) = / if it holds that s,/ = /. These operators, like the ordinary 
divided differences di, satisfy the Coxeter relations (12.61) . so for w £ Soo we may define 


TT w = TT il --- 7 Ti k for any reduced word (sq ,... ,Si k ) £ lZ(w). (3.12) 

We note as the following lemma one less standard property of these operators. 

Lemma 3.39. If i,n £ P and f £ Voo then r n (iTif) = < 1 

1 Jn{f) if i>n. 

Proof. Checking the lemma is a simple exercise in algebra which we leave to the reader. □ 


The next theorem appears as m Eq. (4.25)]. We include a proof for completeness, since this 
result is central to what follows and since [40] is currently out of print and difficult to obtain. 

Theorem 3.40 (Macdonald [ID]). For all / £ V n it holds that stb n (/) = ir Wn f. 

Proof. Define the operator r n = tt\ ■ ■ ■ ir n . Since Xi(njf) = nj(xif) for i < j, it follows that we may 
also write r n f = d\ ■ ■ ■ d n (x\ ■ ■ ■ x n f) for / £ Voo■ Suppose u £ Soo has largest descent at most n. 
We claim that T n & u = &^ u ) where i(u) denotes the permutation 

i(u) = li x u = [ 1 ,7i(l) + 1 , u(2) + 1 ,... ] € Soo- 

To show this, first assume u £ S n and let v = w n+ \w n u = [ri(l) +1, u( 2 ) + 1 ,..., u(n ) + 1,1] £ S n+ 1 . 
Since the product x\ ■ ■ ■ x n is invariant under S n , it holds by Theorem 12.91 that 


x\ ■ ■ ■ x n & u — x\ ■ ■ ■ x n d u ~i Wn & Wn — d u ~i Wn {x\ ■ ■ ■ x n 6 Wn ) — d u -i Wri 6 Wn+1 — & v 
so T n & u = d\ ■ ■ ■ d n (xi ■ ■ ■ x n 6 u ) = d\ ■ ■ ■ d n 6 v . It is clear that we have a descending chain 

v > vs n > vs n s n - 1 > • • • > vs n s n - 1 • • • si = i(u) 

so we conclude by Theorem 12.81 that r n 6 u = &i( u ) when u £ S n . To prove the claim in general, 
observe that 6 U £ V n by Proposition 12.121 so 7 Tj© u = & u for all i > n. Therefore if u £ Sn for 
some N > n, then Tjy&u = x n & u = &i( u ) by the part of the claim already shown. 

Fix / £ V n . Given our claim, it follows by Proposition 12.121 and Corollary 13.381 that stb n (/) = 
r n (T 2 n—l • • • T n+ \T n f) . One checks using Lemma 13.391 that if N > n then r n (jNg) = T n -\r n {g) for 
all g £ Voo- Using this property, we deduce that 


stb n (/) = r n (t 2 „_i • • • T n+ 1 r n f) = T^_ 1 r n (/) = rff_ x f. 

Recall that 7 r f = iii for all i £ P. Therefore, if vj £ W and s £ S, then n w ir s is equal to ir w when 
s £ Des r(w) and to ir ws when s ^ Desft(w). Using this property, it is a simple exercise to check 
that rf_ l = n Wn which suffices to complete the proof. □ 
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We may now begin to say something about the “stability” of the formulas in Theorem 13.271 
In view of the preceding theorem and (13.111) . it follows that stb n (fg) = f stb n (g) if / G Voo is 
invariant under the action of S n . We would like to apply something like this identity to Theorem 
13.271 but, problematically, the involution Schubert polynomials 6 gk and 6 FFF appearing in that 
result are symmetric only under the action of the subgroup Sk x £ n _fc, not all of S n . To get around 
this difficulty, we factor stb n into two operators, one of which respects the partial symmetry which 
we encounter, in the following way. 

Fix nonnegative integers p, q with n = p + q, and write A pxq for the subring of polynomials in 
V n which are fixed by the action of S p x S q C S n . Thus A n = Ao X n = A nx o C A pxq . Let 

g p ,q = W n ■ (w p x w g ) = [q + 1, q + 2,..., n, 1, 2,... , q\ G S n 

and define stb P)9 : V n ->■ A pxq and stb„/ P)9 : A pxq ->• A n by 

stb p> g = TtwpXWq and stb n / p9 = 7r Pp9 . 

Note that it is clear from Theorem 13.401 that stb n (/) = stb n / Pi9 stb Pi9 (/) for all / G V n . 

Lemma 3.41. Let p, q G N and n = p + q. If / G A pxq and g G Voo such that stb P)IJ (g) € A n , then 

stb n (fg) = stb n {f)stb Ptq {g). 

Proof. Our hypotheses together with (13.111) imply that stb Pi9 (/) = / and 

stb p>q (fg) = f ■ stb p>q (g) and stb n / Pi9 (/ • stb Pi9 (p)) = stb n / Pi9 (/) stb Pi9 ( 5 ). 

Hence stb n (fg) = stb n/m stb p , q {fg) = stb n / P , q {f) stb Ptq (g) = stb „(/) stb Piq (g). □ 

The algebra of symmetric functions A may be identified with its graded dual and so given 
the structure of a graded, self-dual Hopf algebra; see [T8J [56] for the details of this standard 
construction. The coproduct A : A —>• A (g> A of this Hopf algebra is defined as the linear map 
satisfying A(/) = Yli9i ® hi for / G A, where <?*, hi G A are symmetric functions such that, when 
written as a function of two countable, commuting sets of variables, / decomposes as 

f{x i,x 2 , ...,yi,y 2 ,...) = ^.g*(x'i,x 2 ,... )h i (y 1 ,y 2 ,.. ■ )• 

i 

For any partition u. it holds that A (s u ) = v c \ where cf are the Littlewood-Richardson 

coefficients. Write u : A —>• A for the linear map with uj(s v ) = s^t for all partitions u. 

Lemma 3.42. Let p, q G N and n = p + q. Suppose w G S q has largest descent at most p. If 

(id ® w) o A (F w ) = A (F w ) 

then stb Pi9 (4> Pi9 (6 w (x;y))) = F w (x i,.. .,x n ). 

Proof. First note using Definition 12.101 and Proposition 12.121 that 

^ p,q y')') — ^ © u (xi, • • •, x p )e„(x p+ i, • • •, x n ) G V n 

W=V~ 1 U 

£(w)=£(u)-\-£(v) 
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since in the right hand sum all indices u have largest descent at most p (as this is the largest 
possible descent of w), while all indices v have largest descent at most q (as v G S q ). Since stb Pi? 
acts on (& w (x-,y)) as the operator vr WpX w q , it follows via Theorem 13.401 that 

Stbj^q (^p,<? (®iu(*^) 2/))) — ^ ' F u (x±, . . . , Xp)F v {Xp-\-\) . . . , X n f (3.13) 

w=v~ 1 u 

£(w)=£(u)+£(v) 


Now, we have from [38], Proposition 5 and Theorem 12] that A (F w ) = ^ F u ® F v where the sum 
is over all u,v G S q with w = vu and £(w) = £(v) + £{v). On the other hand, Macdonald [301 
Corollary 7.22] proves that u(F w ) = F w - 1 . Hence, if (id <g> w) o A (F w ) = A (F w ), then 

F w (xi,X2,...,yi,U2,---) = ^2 F u (x 1 ,X 2 ,...)F v (y 1 ,y 2 ,...). 

W=V~ 1 U 

Z(w)=t(v)-\-£(u) 


On transposing the variables yi and x p +i for i G [q] (which by symmetry does not affect either 
expression) and then setting x n+ i = yi = 0 for i G P, the left side of this identity becomes 
F w (x i,...,x n ) while the right side becomes the formula (13.131) for stb Pi g {$ p ,q (& w (x; y))); these 
expressions are therefore equal when id <8> w fixes A (F w ). □ 

Let p n = x™ + x 2 + • • • G A for n G N denote the usual power sum symmetric function. Since 
A(p n ) = p n <8> 1 + 1 <8> p n and co(p n ) = (—1 ) n ~ 1 p rL (see |T8j Proposition 2.23 and §2.4]), and since A 
and uj are algebra homomorphisms, it follows that (l®w)o A(/) = A(/) whenever / belongs to 
the Hopf subalgebra 

A° dd = %\Pl,P3,P5,---} C A 

generated by the odd-indexed power sum symmetric functions. This subalgebra is studied in a few 
places (see, e.g., [H 123152]), but does not seem to have an established name. The following theorem 
is the main result of this section, and will imply the results described in the introduction. 

Theorem 3.43. Let y G Z(S 00 ) be weakly dominant with k = n(y). If F r ^ G A odd then 

F y = F gk F r[y) and F™ = F™F r(y) 

where the second identity applies only in the case when y is fixed-point-free. 

Proof. Let n be the smallest integer such that y G S n . so that n = 2k when y is fixed-point-free. As 
noted in the preceding discussion, the operator id(8>cu preserves A (F r ^) and so applying Lemmas 
13.411 and 13.421 to Theorem 13.271 shows that F y {xi,...,x n ) = F gk (x\,... ,x n )F r ^(xi,... ,x n ) and, 
when y is fixed-point-free, that F y PF (x±,... ,x n ) = F gk F (xi ,..., x n )F r ^ (aq,..., x n ). It remains 
only to argue that these identities in A n lift to identities in A. 

Write £(A) for the number of parts in a partition A. Let be the subspace of A„ spanned 
by the Schur polynomials sa(xi, ..., x n ) for partitions A with £(A) < k, and likewise define A^ ^ = 
Z-span{sA : ^(A) < k}. It is well-known that the restriction map r n defines a bijection Aqq & —>• A n ^ 
whenever k < n (see, e.g., sn Proposition 1.2.1]) and that fg G A oa j + f : whenever / G A^j and 
g G Aqq^ (see )4 11 §1.5.4]). Hence, if we have (/, g, h) G A^j x A^^ x A ooj+k an d j + k < n, then 

h{x\,... ,x n ) = f(xi,... ,x n )g(xi,... ,x n ) G A n h = fg£ A (3.14) 
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since we may obtain the right identity by applying the inverse of the bijection r n : J^oo,j+k ~> A-nj+k 
to both sides of the equation on the left. 

It follows from m Theorem 4.1] that F u £ A^fc if u £ Sqq has largest descent at most k. In 
view of Proposition 12.41 we thus have F u £ (respectively, F fpf £ ^-co,k) whenever u is an 

involution (respectively, fixed-point-free involution) with largest descent at most k. Since 2 k < n 
and y £ S n , and since gk and r(y) both have largest descent at most k, we may apply ()3.14l) to 
deduce the desired identities from the formulas in the first paragraph. □ 

Recall that 6 n = (n — 1, n — 2,..., 2,1). The subalgebra A odd C A has a distinguished basis 
{P\} indexed by strict partitions (see [1][52|), called the Schur P-functions. An element / £ A odd 
is Schur P-positive if it is a nonnegative linear combination of Schur P- functions. 

Theorem 3.44. Let y £ Z(Soo) be weakly dominant with k = n(y). Suppose D(r(y )) is equivalent 
to a skew shape of the form 6 m / A for some m £ P and partition A C 5 m . Then 

Fy = s s k+1 s S m/\ and 4 FPF = S 6k S Sm /A 

where the second identity applies only in the case when y is fixed-point-free. Moreover, in this case 
the symmetric functions F y and (when defined) F fpf are Schur P-positive. 

Remark. It will be shown in [24] that F y is Schur P-positive for all y £ X(S' 00 ), but only in the 
special case just described does this follow directly from our present methods. It is still an open 
question whether F fpf for z £ X FPF (S , 00 ) is similarly Schur P-positive. 

Proof. By Proposition 12.171 we have P r ( y ) = s 8 m hj.- and it is well-known that this skew Schur 
function belongs to A odd : see Proof 2 of |46l Corollary 7.32], or just adapt the argument in [511 
Proposition 7.17.7]. From this, the formulas for F y and F fpf are immediate by Proposition 13.351 
and Theorem 13.431 For the last assertion, we note that skew Schur functions of the form sg rn / y are 
Schur P-positive (see HI), and that Schur P-positivity is closed under products (see [52, §8]). □ 

The most important special case of the preceding result is Theorem 11.41 from the introduction, 
whose proof we now give. 

Proof of Theorem \1.4\ One checks that n{w n ) = [n/2j and r(w n ) = w\ n / 2 \- We have seen that 
D(w n ) is the shape of S n , so by Theorem 13.441 F Wn = ss k+ 1 ss m for k = [^J and m = [~^~| and 
F FPF = ( ss n ) 2 . The theorem follows as {k + 1, m} = {p, q} for p = 1"^^-] and q = • □ 


A Geometric complements 


Throughout this appendix, we define (3 and K C GL n (C) as in Section 13.41 and fix a weakly 
dominant involution z £ Zk with p = k(z) and set q = n — p. It is convenient to define 


&' = 


2?6 g 

p;FPF 

9p 


if (3 is symmetric 
if j3 is skew-symmetric 


and 6 " = $ p , q (& r ( z )(x;y)). 


We outline here a geometric proof of Theorem 13.271 

That statement asserts [Y^] = & ■ 6 ", so it is natural to ask whether is it possible to write 
as a reasonable intersection of two subvarieties of Fl(n) with cohomology classes 6 ' and 6 ". As 
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stated this question cannot have an affirmative answer, since the elements of the cohomology ring 
H*(F\(n), Z) which are classes of subvarieties are exactly the nonnegative integer combinations of 
Schubert classes (cf. [33 , Remark 2.1.2]), which include & but not necessarily 6". 

Instead, define Fl(/c,n) as the variety of partial flags F, = (0 = Fq C F\ C • • • C C C n ), 
with each Fj being an z-dimensional subspace of C n , and consider the split flag variety 

Fl sp (; p,q) = {( F.,G .) G Fl(p,n) x F\(q,n) : F p n G q = 0}. 

Let 7r be the projection Fl sp (p, q) —>■ Fl(n) defined by 

Tt(F,, G ,) = (Fi C • • • C F p C F p © G\ C • • • F p © Gq). 

It suffices by the following lemma to study the question posed in the previous paragraph in Fl sp (p, q) 
rather than Fl(ra). 

Lemma A.l. The induced map n* : iL*(Fl(n),Z) —>• iL*(Fl sp (p, q), Z) is an isomorphism. 

Proof. The proof is similar to arguments discussed in [9] §3.6.13]. Let Gr(g, C n ) denote the Grass- 
mannian of g-planes in C n . Then 7r is the projection of a fiber bundle, whose fiber over H, is 



/ 

* ... * i .. 

• o' 


{V G Gr(g, C n ) : V n H p = 0} ~ < 

rowspan 

>« 

... o 

... * 

• • • * 
_i 

• l 



These fibers are contractible, and we claim that this makes n a homotopy equivalence. One way to 
see this is to use the long exact sequence of a fibration to first conclude that 7 r is a weak homotopy 
equivalence. Both Fl(n) and Fl sp (p, q ) are smooth manifolds (the latter because it is an open subset 
of the smooth manifold Fl(p, n) x Fl(g, n)) and every smooth manifold has the homotopy type of a 
CW-complex via Morse theory |421 Corollary 6.7]. By Whitehead’s theorem |211 Theorem 4.5], 7 r 
is a homotopy equivalence. □ 

To prove Theorem 13.271 geometrically, we will exhibit subvarieties Z, Z', and Z" of FI sp (p,q) 
whose cohomology classes correspond to [Y^], &, and 6 " under the isomorphism n*, and which 
are such that the intersection Z' n Z" is generically transverse and equal to Z. These hypotheses 
imply [Z\ = \Z'\\Z"\ and by passing back to Fl(n), we will be able to conclude that Theorem 13.271 
holds in the cohomology ring iL*(Fl(n), Z). 

To this end, define Z' and Z" as the subvarieties of Fl sp (p, q) given by 

Z' = {( F.,G .) € Fl sp (p,g) : fflF p ,F p ) = 0} 

Z" = {(F.,G.) € Fl sp (p, q) : rank(^| FiXG .) < rk r ( z) {i,j) for (i,j) € \p\ x [g]}. 

We also define an open subset Z" of the second variety by replacing the inequality < by = in each 
rank condition. The following alternative description of Z" will be useful. 

Observation A.2. The rank of fl\ FixGj is equal to the rank of the Gram matrix \/3(fi, 9j)](i,j)G[p]x[q]- 

For the rest of this section we write ?k(z) f° r tb e length given by l(z) when I\ = O (n) and by 
^fpf(~) when K = Sp(n). Note that £k(z) is then the codimension of in Fl(n). 
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Lemma A.3. Set-theoretically, we have Z' D Z" = n 1 (Yf c ) and Z' n Z" = n 1 (Yj < ). 

Proof. We consider only the case when K = 0(n), since the symplectic case follows in the same 
way. Let C\j denote the set of F, £ Fl(n) such that rank(/3|p iX F,) < rk It follows by 
Proposition 13. 161 and Lemma 13.231 that 7 t~ 1 (Y z k ) = Y' n Y" where 

Y'= f| vr ■~ 1 (C ij ) and Y" = f| Tr" 1 ^)- 

(i,j)£D(g P ) (iJ)€ E p(r(z)) 


If ('<, j) £ D(g p ), so that i.j < p, then 

7T _1 ( Qj) = {( F,,G .) € FI sp {p,q) : rank(fl\ FiXFj ) < rk z (i,j)}. 

Since rk z (i,j) = 0 whenever i , j < p, it follows that Y' = Z'. Suppose on the other hand that 
(i, j) £ E p (r(z)), so that j < p < i. Then 

= {(F.,G.) £ FI sp {p,q) : vank(P\ {Fp(BG ._ p)xF .) < rk z (i,j)}. 

If (. F.,G .) £ Z' then rank(/3| (Fp@G ._ )xF ) = rank(/3| Gi _ pXjF .) since f3(F p ,F p ) = 0. As rk z (i,j) = 
rk r (z){j->i ~ p) when i > p by Lemma 13.231 it follows that 

Z' FI 7r _1 (C'ij) = {(F„G.) £ Z' : rank(/3| Gi _ pXjF .) < rk z (i,j)} 

= {( F.,G .) £ Z' : rank(/3| Gi _ pXF .) < rk r {z)(jP ~ P)} 

= {( F.,G .) £ Z' : rank(/3| FjXGi _ p ) < rk r{z) (j,i - p)}. 

Combining these observations, we see that tt~ 1 (Y z k ) = D(j j)eE p (r(z)) Z' Cvr -1 (Cy) = Z'C\Z". These 
arguments go through just as well if we replace the inequalities < by = in each rank condition, so 
we conclude that 7r _1 (y/ f ) = Z' Fl Z" also. □ 

Lemma A.4. The varieties Z' and Z" intersect generically transversely; that is, they intersect 
transversely on a Zariski-open dense subset of Z' n Z". 

Proof sketch. It will be convenient to work at first in the full product X = Fl(p, n) x Fl(q, n) 
rather than Fl sp (p, q ). To that end, we define subvarieties r ; , r // , and T ,r of X using the same rank 
conditions as Z', Z", and Z" but without the restriction that F p n G q = 0. Given G . £ Fl(g,n), 
dehne 

V'g. = {F. £ Fl(p, n) : (F„ G .) € r'} and V£ m = {F. £ Fl(p, n) : (F„ G.) £ f"}. 

It is not hard to show from the definition of transversality that if V G and Vff are transverse at 
(F m ,G m ), and so are the analogous varieties where the first coordinate is fixed at F,, then T 7 and 
f" are transverse at (F,,G.). The second condition is easy to check, so we focus on transversality 
of V' G% and V Gm ■ 

Let h* denote the adjoint of h £ GL n (C) with respect to the form fl. It is a simple consequence 
of the relevant rank conditions that V G h = V G . The varieties Y' Gt and V G are smooth 

because they are homogeneous spaces for K and a Borel subgroup depending on G . respectively. 
Thus we can apply Kleinian’s transversality theorem m and conclude that, having fixed G ,, there 
is a Zariski-open set of h for which V G and V G intersect transversely in the expected dimension. 
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This leads to an open set U C FI sp (p,q) on which Z" and Z' intersect transversely, and calculating 
the expected dimension of intersection shows that they do in fact intersect on U. 

By Lemma IA.31 Z' fl Z," = 7 r 1 {Y Z K ). Because V' G% is ii-stable, we can assume that U is K- 
stable. Since U ri7r _1 (Y^) ^ 0 and K acts transitively on 7r _1 (Y Z K ), we see that U intersects every 
component of 7 t _ 1 (Y^), and therefore every component of its closure Thus U D7r —1 (Y z ^) 

is an open dense subset of □ 

Lemma A.5. Relative to Fl sp (p, q), it holds that codim Z' = £k:( g p ) and codim Z" = £(r(z)). 

Proof. Under the map Fl sp (p, q ) —> Gr (p, C n ) defined by ( F,,G ,) i —> F p , the variety Z' is the inverse 
image of the orthogonal (if /3 is symmetric) or isotropic (if /3 is skew-symmetric) Grassnrannian 

{UGGr(p,C"):/?(V,U) = 0}, 

which one checks has codimension ixidp) in Gr(p, C n ). Since (P # ,G») >-)■ F p is the projection of a 
fiber bundle, Z' has the same codimension. 

Likewise, since 7r is the projection of a fiber bundle, codim 7r _1 (Y, A ) = codim = £k{z). By 
Lemmas IA.3I and IA.41 the intersection Z' fl Z" is generically transverse and equal to 7r~ 1 (Y 2 ft "), so 

£k(z) = codim 7r _1 (Y. A ) = codim Z' + codim Z" = Ixidp) + codim Z". 

Hence, codim Z" = £k(z) — £k( 9 p) = ^(r(z)). □ 

Next, we compute the cohomology classes of Z' and Z" by interpreting them as degeneracy 
loci, according to the following general setup. Suppose we have two vector bundles P and Q over a 
smooth complex variety X and a bundle map / : P —>• Q. We are interested in the locus L of points 
x £ X where f x : P x —>• Q x satisfies some kind of rank conditions, and P x denotes the fiber of P 
over x. We will compute the classes of Z' and Z" by realizing them as such loci L and appealing 
to theorems which give universal formulas for [L\ in terms of Chern classes of P and Q. 

Let 0 C Pi C • • • C P n —)■ X be a complete flag of vector bundles over a smooth complex variety 
X, so that rankP,; = i. Dually, let X Q n Q n -1 -» • • • -» Qi -» 0 be a sequence of surjections 
of vector bundles with rankQj = i. Suppose / : P n —> Q n is a bundle map. Given v G S n , let L v be 
the locus where rank(/ : Pi —> Qj) < rk v (i,j)', this construction covers all the rank conditions one 
could impose on / which give non-empty loci, cf. m Lemma 3.1]. Write Ci(S) for the i th Chern 
class of S, an element of H 2l (X, Z). Finally, define Xj = ci(kerTj -» I)_i) and yi = ci(Pj/Pj_i). 
Fulton [[15] gives this formula for [L v \ in terms of double Schubert polynomials: 

Theorem A.6 (See [15]). If codim L v = £(v), then [L v \ = <3 v (x-y ) 6 H 2 ^ V \X). 

Say that a polynomial / € V n represents an element of P*(Fl sp (p, q),Z) if it represents the image 
of this element under the map given by composing the inverse of 7r* with the Borel isomorphism. 

Lemma A.7. 6 " represents the cohomology class \Z"\ G H*(Fl sp (p, q), Z). 

Proof. Let T, be the tautological flag of vector bundles on Fl(ra), so that the fiber of 7) over P. is 
Pj. Upon pulling back to Fl sp (p, q) via 7r, T splits into two flags of bundles, 0 = JJq C U\ C • • • C 
U p = n*T p and 0 = Vq C V\ C • • • C V q , where the respective fibers of Ui and V) over (P., G m ) are 
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Fi and Gj. Under the Borel isomorphism discussed in Section 1231 Xi E LP(Fl(n),Z) corresponds 
to ci((Tj/Tj_i)*) = -a(Ti/Ti- 1 ). Since 


7 FlTi/T^) 


Ui/Ui _i if i<p 
Vi-p/Vi-p -1 if i > p, 


the functoriality of Chern classes shows that 7 r *{xi) is —c\(Ui/Ui-\) or —ci(V)_ p /V)_ p _i) as appro¬ 
priate. By Lemma lA.il 7r* is an isomorphism, and we identify x t with n*(xj.) E H*(Fl sp (p, q), Z) in 
the rest of this section. 

Define a bundle map / : V q U* on the fiber over (F,,G,) by f(v) = j3(y,-)\ f p - Then Z" 
is the locus where rank(/ : V) JJ* ) < rk r ( z )(i,j). Since codim Z" = £(r(z)) by Lemma [A. 51 
Theorem IA. 6 I gives [Z"\ = 6 r ^(u - ,v) where 

Ui = ci(ker(U* -» U*_ t )) = ci((C/;/[/;_i)*) = Xi and = ci(U;/U_i) = -x k +i 


which means precisely that \Z"\ = 6". □ 

To compute [Z'\ we use a degeneracy locus result for bundle maps with some symmetries. Say 
that a linear map g : V —> V* from a vector space to its dual is symmetric or skew-symmetric if 
the associated bilinear form (v,w) *—> g(v)(w) is symmetric or skew-symmetric. 

Theorem A.8 (See [20]). Suppose P is a rank n vector bundle over a smooth complex variety X, 
and g : P —> P* is a bundle map. Let L r C X be the locus where g has rank at most r. 

(a) If g is symmetric and L r has codimension ( n-_ 9 +1 ), then 

[L r \ — 2 det [c n - r -2i+j+l (P )]l 


(b) If g is skew-symmetric and r is even and L r has codimension ( n 9 r ), then 

[L r \ — det[c n -r-2i+j{P )] l<i ,j<n— r— 1 • 


Lemma A.9. & represents the cohomology class \Z'\ E H*(Fl sp (p, q), Z). 

Proof. Let U, V, and x\,...,x n be as in the proof of Lemma [A. 71 Define g : U p — > U* on the 
fiber over (F,,G 9 ) by g{v) = f3(v,-)\ f p - Then g is symmetric (if K = O(n)) or skew-symmetric (if 
K = Sp(n)), and Z' is the locus where g has rank zero. By Lemma I A. 51 codim Z' is the expected 
codimension required for Theorem IA.81 so 


[Z'] 


2 n det[cp-2i+j+i(U*)\i<ij<p if K = O(n) 
det [cp- 2 i+j(U* )]i<jj<p-i if K = Sp(n). 


We have U* ~ ®" = i(C/j/?7j-i)*, so the Whitney sum formula gives Ci(U*) = e;(x i, ... ,x p ). The 
Jacobi-Trudi identity alongside Proposition 13. 101 now shows that \Z'\ = &. □ 

Remark. As discussed in [53], involution Schubert polynomials also arise from degeneracy locus 
formulas. Suppose 0 C Pj C ■ • • C P n is a complete flag of vector bundles over a smooth complex 
variety X. Assume P n is equipped with a non-degenerate bilinear form j3 which is either symmetric 
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or skew-symmetric. Given y £ Ik where K is the group of automorphisms of /3 on a fiber of P n , 
define the locus 


L y = jx G A : rank (/^(p^xfP,-)*) < rk y (i,j) for all 1 < i,j < n| . 

Let Xj be the Chern class ci(Pj/Pj_i). It is shown in j54j that if /3 is suitably generic, then 


l^y] ~ 


2 K ^& y (xi ,..., x n ) if f3 is symmetric 
© FPF (xi, ..., x n ) if j3 is skew-symmetric. 


We can now reprove Theorem 13.271 Since n is the projection of a fiber bundle, 7r*[Y, A ] = 
[7 t~ 1 (YJ < )\ (cf. [TBJ §B. 1]), and so by Lemmas I A. 3 1 and I A. 41 we have 

tt*[Y z k ] = \Z'][Z"]. 


By Lemmas I A. 71 and lA.9l passing back to P*(Fl(n), Z) via (n*) F , while recalling our identification 
of 7 r*(xj) with Xj, gives 

[Y z k ] = 6' • 6". (A.l) 

This proves Theorem 13.271 in FP(Fl(n), Z), which, however, is weaker than the equality of poly¬ 
nomials given by Theorem 13.271 To deduce the latter, it suffices to show that 6 ' ■ &" is in 
S = Z-span{G w : w € S n }, since these Schubert polynomials remain linearly independent in 
P*(Fl(n),Z), and & z is also in S. As per [16], §10.4], 

S = Z-spanjx^ 1 ■ ■ • x l ™ : 0 < ij < n — j for each j}. 

Thus, if x ^ 1 • • • x l ™ is a monomial in &, then ij <p — j for j < p while ij = 0 for j > p. Similarly, 
one checks using Definition 12. 101 that for a monomial in <3", ij < n — p — j for j < p while ij < n — j 
for j > p. Combining these, for a monomial in & ■ 6" one has ij < n — 2 j for j < p while ij < n — j 
for j > p. In particular, 6' • &' £ S. 
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